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FOREWORD 


The Software Engineering Laboratory (SEL) is an organization 
sponsored by the National Aeronautics and Space Administra- 
tion/Goddard Space Flight Center (NASA/GSFC) and created for 
the purpose of investigating the effectiveness of software 
engineering technologies when applied to the development of 
applications software. The SEL was created in 1976 and has 
three primary organizational members: 

NASA/GSFC, Systems Development Branch 

The University of Maryland, Department of Computer 
Science 

Computer Sciences Corporation, Systems Development 
Operation 

The goals of the SEL are (1) to understand the software de- 
velopment process in the GSFC environment; (2) to measure the 
effect of various methodologies, tools, and models on this 
process; and (3) to identify and then to apply successful 
development practices. The activities, findings, and 
recommendations of the SEL are recorded in the Software 
Engineering Laboratory Series, a continuing series of 
reports that includes this document. 

The major contributors to this revision of this document are 

Linda Morusiewicz (Computer Sciences Corporation) 

Jon Valett (NASA/GSFC) 

Single copies of this document can be obtained by writing to 

Systems Development Branch 
Code 552 

Goddard Space Flight Center 
Greenbelt, Maryland 20771 
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ABSTRACT 
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Th i-s~doc. umen.t — is an annotated bibliography of technical 
papers, documents, and memorandums produced by or related to 
the Software Engineering Laboratory./ More than 100 publica- 
tions are summarized. These publications cover many areas 
of software engineering and range from research reports to 
software documentation. 


Th~i-s-.document.Jias been- updated and- reorganized" sub’st ant i a 1 ly 
sjince the original .version (SEL-82-606 , November 1982).. All 
materials have been grouped into eight general subject areas 
for easy reference: 



/ 

/ 
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The Software Engineering Laboratory ' 

J 

The Software Engineering Laboratory: Software De- 

velopment Documents . 

Software Tools 

Software Models 

Software Measurement \ 

Technology Evaluations ^ 

£ 

Ada Technology . 

Data Collection « 


Subject and author indexes further classify these documents 
by specific topic and individual author. 
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SECTION 1 - INTRODUCTION 


This document is an annotated bibliography of technical 
papers, documents, articles, and memoranda produced by or 
related to the Software Engineering Laboratory (SEL) . It is 
intended to provide a quick reference to the published re- 
sults of SEL research and development activities. 

More than 100 publications are summarized in this document. 
Each summary includes the size of the publication (number of 
pages), a description (abstract) of its contents, and its 
original citation. Previous versions and subsequent re- 
printings are also identified where appropriate. 

The publications described here cover many aspects of soft- 
ware engineering and range from research reports to software 
documentation. They are divided into eight general subject 
areas: 

• The Software Engineering Laboratory 

• The Software Engineering Laboratory: Software De- 

velopment Document 

• Software Tools 

• Software Models 

• Software Measurement 

• Technology Evaluations 

• Ada Technology 

• Data Collection 

Two indexes are included at the end of this document to as- 
sist in identifying materials by subject and author. In the 
subject index, a list of the abstract numbers of related 
publications follows each subject heading. For the author 
index, publications with their corresponding abstract num- 
bers are ordered by individual authors. 
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Copies of individual publications listed in this bibliog- 
raphy can be obtained from one or more of the sources shown 
in Table 1-1. The acronyms defined in the table appear 
after each abstract and indicate the document's availabil- 
ity. Any material not labeled with one of these acronyms 
can be obtained only from the author(s) . 
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Table 1-1. Availability of SEL Literature 


Acronym 


Source 


Address 


SDB 


CAS I 
NTIS 
JAO 


Systems Development 
Branch 


NASA Center for Aerospace 
Information 1 (and source 
above) 

National Technological 
Information Service 2 
(and sources above) 

Journals and other 
private publishers 


Code 552 

Goddard Space Flight 
Center 

Greenbelt, Maryland 
20771 

P. 0. Box 8757 

BWI Airport, Maryland 

21240 

5285 Port Royal Road 
Springfield, Virginia 
22161 

See specific citation 


^Open to Federal Government agencies only at no charge. 
2 

There is a per-page charge for reprinting documents. 
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2.1 COLLECTED SOFTWARE ENGINEERING PAPERS: VOLUME I 

SEL-82-004, JULY 1982, 118 PAGES 


This document is a collection of technical papers prepared 
by SEL participants during the 5-year period ending 
December 31, 1981. The 10 papers are organized into 4 major 
topics and each paper's section number within this Annotated 
Bibliography is provided for further reference. NTIS 


Technical Paper 


Section 


THE SEL ORGANIZATION 

"The Software Engineering Laboratory: Objectives," 

V. R. Basili and M. V. Zelkowitz 

"Operation of the Software Engineering Laboratory," 
V. r. Basili and M. V. Zelkowitz 

RESOURCE MODELS 

"Resource Estimation for Medium-Scale Software 
Projects," M. V. Zelkowitz 

"A Meta-Model for Software Development Resource 
Expenditures," J. W. Bailey and V. R. Basili 

"Can the Parr Curve Help With Manpower Distribution 
and Resource Estimation Problems?" V. R. Basili and 

J. Beane 

SOFTWARE MEASURES 

"Measuring Software Development Characteristics in 
the Local Environment," V. R. Basili and 
M. V. Zelkowitz 

"Programming Measurement and Estimation in the 
Software Engineering Laboratory," V. R. Basili and 

K. Freburger 

"Evaluating and Comparing Software Metrics in the 
Software Engineering Laboratory," V. R. Basili and 
T. Phillips 

SOFTWARE ENGINEERING APPLICATIONS 

"Models and Metrics for Software Management and 
Engineering," V. R. Basili 

"Use of Cluster Analysis To Evaluate Software 
Engineering Methodologies," E. Chen and 
M. v. Zelkowitz 


2.25 

2.26 


5 . 2 


5.8 


5.9 


6.3 

6.5 

6 . 6 


5.5 


7.7 
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2.2 COLLECTED SOFTWARE ENGINEERING PAPERS: VOLUME II . 

SEL-83-003 , NOVEMBER 1983, 100 PAGES 

This document is a collection of technical papers prepared 
by SEL participants from January 1, 1982, through 
November 30, 1983. The nine papers are organized into four 
major topics and each paper’s section number within this 
Annotated Bibliography is provided for further reference. 
NTIS 


Technical Paper Section 

THE SOFTWARE ENGINEERING LABORATORY 

"Measuring Software Technology," W. W. Agresti, 6.10 

F. E. McGarry, D. N. Card, et al. 

"Technical Summary 1982: Report to the National Other 

Aeronautics and Space Administration," V. R. Basili Refer- 
ences 

RESOURCE MODELS 

"Comparison of Regression Modeling Techniques for 5.11 

Resource Estimation," D. N. Card 

"Early Estimation of Resource Expenditures and 6.7 

Program Size," D. N. Card 

SOFTWARE MEASURES 

"Metric Analysis and Data Validation Across 6.9 

FORTRAN Projects," V. R. Basili, R. W. Selby, and 
T. Phillips 

"Monitoring Software Development Through Dynamic 5.12 

Variables," C. W. Doerflinger and V. R. Basili 

"Software Errors and Complexity: An Empirical 6.11 

Investigation," V. R. Basili and B. T. Perricone 

DATA COLLECTION 

A Methodology for Collecting Valid Software 9.7 

Engineering Data . V. R. Basili and D. M. Weiss 

"Data Collection and Evaluation for Experimental 9.6 

Computer Science Research," M. V. Zelkowitz 
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2.3 COLLECTED SOFTWARE ENGINEERING PAPERS: VOLUME III . 
SEL-85-003, NOVEMBER 1985, 132 PAGES 

This document is a collection of technical papers prepared 
by SEL participants from November 30, 1983, through 
November 1, 1985. The 12 papers are organized into 3 major 
topics and each paper's section number within this Annotated 
Bibliography is provided for further reference. NTIS 

Technical Paper Section 

THE SOFTWARE ENGINEERING LABORATORY 

"A Software Technology Evaluation Program," 7.18 

D. N. Card 

"A Methodology for Collecting Valid Software 9.8 

Engineering Data,” V. R. Basili and D. M. Weiss 

TECHNOLOGY EVALUATION 

"Quantitative Evaluation of Software Methodology," 7.17 

V. R. Basili 

"Four Applications of a Software Data Collection 7.16 

and Analysis Methodology," V. R. Basili and 
R. W. Selby, Jr. 

"Measuring the Impact of Computer Resource Quality 7.12 

on the Software Development Process and Product," 

F. E. McGarry, J. Valett, and D. Hall 

"Analyzing the Test Process Using Structural Cover- 7.11 
age," J. Ramsey and V. R. Basili 

"A Practical Experience With Independent Verifi- 7.10 

cation and Validation," G. Page, F. E. McGarry, 
and D. N. Card 

"ARROWSMITH-P — A Prototype Expert System for Soft- 7.26 

ware Engineering Management," V. R. Basili and 

C. L. Ramsey 

SOFTWARE MEASUREMENT 

"Finding Relationships Between Effort and Other 5.15 

Variables in the SEL," V. R. Basili and 
N. M. Panlilio-Yap 

"Calculation and Use of an Environment's Charac- 6.17 

teristic Software Metric Set," V. R. Basili and 
R. W. Selby, Jr. 

"Criteria for Software Modularization," 6.16 

D. N. Card, G. T. Page, and F. E. McGarry 
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Technical Paper 


Section 


"Evaluating Software Development by Analysis of 
Changes: Some Data From the Software Engineering 

Laboratory," D. M. Weiss and V. R. Basili 


6.18 
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2.4 COLLECTED SOFTWARE ENGINEERING PAPERS: VOLUME IV . 

SEL-8 6-004, NOVEMBER 1986, 108 PAGES 

This document is a collection of technical papers prepared 
by SEL participants from November 1, 1985, through 
September 30, 1986. The six papers are organized into three 
major topics and each paper's section number within this 
Annotated Bibliography is provided for further reference. 
NTIS 


Technical Paper Section 

THE SOFTWARE ENGINEERING LABORATORY 

"Experimentation in Software Engineering," 5.16 

V. R. Basili, R. W. Selby, Jr., and D. H. Hutchens 

"An Approach for Assessing Software Prototypes," 7.20 

V. E. Church, D. N. Card, W. W. Agresti, and 
Q. L. Jordan 

"Towards a General Object-Oriented Software Devel- 8.3 

opment Methodology," E. Seidewitz and M. Stark 

TECHNOLOGY EVALUATION 

"An Empirical Study of Software Design Practices," 7.19 

D. N. Card, V. E. Church, and W. W. Agresti 

"Designing with Ada for Satellite Simulation: A 8.2 

Case Study," W. W. Agresti, V. E. Church, 

D. N. Card, and P. L. Lo 

SOFTWARE MEASUREMENT 

A Study on Fault Prediction and Reliability 5.17 

Assessment in the SEL Environment , V. R. Basili 
and D. Patnaik 
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2.5 COLLECTED SOFTWARE ENGINEERING PAPERS: VOLUME V . 

SEL-87-009, NOVEMBER 1987, 284 PAGES 

This document is a collection of technical papers prepared 
by SEL participants from September 1, 1986, through 
January 1, 1988. The 16 papers are organized into 3 major 
topics and each paper's section number within this Annotated 
Bibliography is provided for further reference. NTIS 


Technical Paper Section 

SOFTWARE MEASUREMENT AND TECHNOLOGY STUDIES 

"A Summary of Software Measurement Experiences in 6.28 

the Software Engineering Laboratory,” J. D. Valett 
and F. E. McGarry 

"Evaluating Software Engineering Technologies," 7.23 

D. N. Card, F. E. McGarry, and G. T. Page 

"Resolving the Software Science Anomaly," D. N. Card 5.18 
and W. W. Agresti . 

"A Controlled Experiment on the Impact of Software 6.20 

Structure on Maintainability," H. D. Rombach 

"An Evaluation of Expert Systems for Software En- 7.21 
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"Comparing the Effectiveness of Software Testing 7.25 

Strategies," V. R. Basili and R. W. Selby 

MEASUREMENT ENVIRONMENT STUDIES 

"Tailoring the Software Process to Project Goals 5.19 

and Environments," V. R. Basili and H. D. Rombach 

"TAME: Tailoring an Ada™ Measurement Environ- 8.7 

ment," V. R. Basili and H. D. Rombach 

"TAME: Integrating Measurement Into Software En- 6.21 

vironments," V. R. Basili and H. D. Rombach 

A Meta Information Base for Software Engineering . 5.21 

L. Mark and H. D. Rombach 

Characterizing Resource Data: A Model for Logical 5.24 


Association of Software Data . D. R. Jeffery and 
V. R. Basili 

ADA TECHNOLOGY STUDIES 

"Measuring Ada for Software Development in the 8.13 

Software Engineering Laboratory (SEL)," 

F. E. McGarry and W. W. Agresti 
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Technical Paper Section 


"General Object-Oriented Software Development: 8.14 

Background and Experience," E. Seidewitz 

"Towards a General Object-Oriented Ada Lifecycle," 8.5 

M. Stark and E. Seidewitz 

"A Structure Coverage Tool for Ada™ Software Sys- 8.6 

terns," L. Wu, V. R. Basili, and K. Reed 

"Lessons Learned in Use of Ada™-Oriented Design 8.8 

Methods," C. E. Brophy, W. W. Agresti, and 
V. R. Basili 
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2.6 COLLECTED SOFTWARE ENGINEERING PAPERS: VOLUME VI . 

SEL-88-002, NOVEMBER 1988, 153 PAGES 

This document is a collection of technical papers prepared 
by SEL participants from June 1, 1987, through January 1, 
1989. The 12 papers are organized into 3 major topics and 
each paper's section number within this Annotated Bibliog- 
raphy is provided for further reference. NTIS 


Technical Paper Section 


SOFTWARE MEASUREMENT AND TECHNOLOGY STUDIES 

"The Effectiveness of Software Prototyping: A 7.22 

Case Study," M. V. Zelkowitz 

"Measuring Software Design Complexity," D. N. Card 6.25 

and W. W. Agresti 

"Quantitative Assessment of Maintenance: An In- 7.24 

dustrial Case Study," H. D. Rombach and V. R. Basili 

"Resource Utilization During Software Development,” 6.22 
M. V. Zelkowitz 

MEASUREMENT ENVIRONMENT STUDIES 

"Generating Customized Software Engineering Inf or- 5.22 
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Software Environments," V. R. Basili and 
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"Validating the TAME Resource Data Model," 6.23 

D. R. Jeffery and V. R. Basili 

ADA TECHNOLOGY STUDIES 

"Experiences in the Implementation of a Large 8.17 

Ada Project," S. Godfrey and C. Brophy 

"General Object-Oriented Software Development with 8.16 
Ada: A Life Cycle Approach," E. Seidewitz 

"Lessons Learned in the Implementation Phase of a 8.15 

Large Ada Project," C. E. Brophy, S. Godfrey, 
w. w. Agresti, and V. R. Basili 

"Object-Oriented Programming in Smalltalk and 8.12 

Ada," E. Seidewitz 
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2.7 COLLECTED SOFTWARE ENGINEERING PAPERS: VOLUME VII . 

SEL-89-006 , NOVEMBER 1989, 157 PAGES 

This document is a collection of technical papers prepared 
by SEL participants from December 1988 through October 
1989. The seven papers are organized into three major 
topics and each paper's section number within this Anno- 
tated Bibliography is provided for further reference. NTIS 


Technical Paper Section 

SOFTWARE MEASUREMENT AND TECHNOLOGY STUDIES 

Establishing a Measurement Based Maintenance Im- 6.29 

provement Program: Lessons Learned in the SEL . 

H. Rombach and B. Ulery 

Maintenance = Reuse-Oriented Software Development . 5.26 

V. Basili 

Software Development: A Paradigm for the Future . 5.27 

V. Basili 
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Integrating Automated Support for a Software Man- 6.30 

agement Cycle into the TAME System . T. Sunazuka and 
V. Basili 

Towards A Comprehensive Framework for Reuse: A 5.25 


Reuse-Enabling Software Evolution Environment . 

V. Basili and H. Rombach 
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"Evolution of Ada Technology in a Production Soft- 8.22 
ware Environment," F. McGarry, L. Esker, and 
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"Using Ada to Maximize Verbatim Software Reuse," 8.23 

M. Stark and E. Booth 


8978 


2-10 


2.8 COLLECTED SOFTWARE ENGINEERING PAPERS: VOLUME VIII 

SEL-90-005, NOVEMBER 1990, 125 PAGES 


This document is a collection of technical papers prepared 
by SEL participants from November 1989 through October 1990. 
The seven papers are organized into four major topics and 
each paper’s section number within this Annotated Bibli- 
ography is provided for further reference. NTIS 

Technical Paper Section 


SOFTWARE MEASUREMENT STUDIES 

"Design Measurement: Some Lessons Learned," 6.31 

H. Rombach 
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ADA TECHNOLOGY STUDIES 

"On Designing Parametrized Systems using Ada," 8.27 

M. Stark 

"PUC: A Functional Specification Language for 8.28 

Ada," P. Straub and M. Zelkowitz 

"Software Reclamation: Improving Post-Development 8.26 

Reusability," J. Bailey and V. Basili 
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SEL-91-005, NOVEMBER 1991, 199 PAGES 

This document is a collection of technical papers prepared 
by SEL participants from November 1990 through November 
1991. The eight papers are organized into three major 
topics and each paper's section number within this Annotated 
Bibliography is provided for further reference. NTIS 


Technical Paper Section 
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"Software Reuse: A Key to the Maintenance 5.30 

Problem," H. D. Rombach 

Support for Comprehensive Reuse . V. R. Basili and 5.31 
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2.10 PROCEEDINGS FROM THE FIRST SUMMER SOFTWARE ENGINEERING 
WORKSHOP . SEL-76-001 , AUGUST 1976, 194 PAGES 

This document reproduces the presentations made by partici- 
pants at the First Summer Software Engineering Workshop held 
on August 5, 1976, at GSFC. The general topic of the con- 
ference was software design. The presentations were grouped 
into the following panels: 

• Requirements analysis and design methodologies 

• Program design languages 

• Automated software tools 

Papers related to the SEL are 

• V. Basili (University of Maryland), "Program Design 
Languages" 

• E. Damon (NASA/GSFC) , "DOMONIC As a Design and Man- 
agement Tool" 

• M. Zelkowitz (University of Maryland) , "Automated 

Tools" 

This first workshop surveyed available state-of-the-art 
software development techniques. The specific applicability 
of the techniques to the GSFC environment was also con- 
sidered. Approximately 25 persons attended this meeting. 
NTIS 
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2.11 PROCEEDINGS FROM THE SECOND SUMMER SOFTWARE ENGINEERING 
WORKSHOP . SEL-77-002 , SEPTEMBER 1977, 146 PAGES 


This document reproduces the presentations made by partici- 
pants at the Second Summer Software Engineering Workshop 
held on September 19, 1977, at GSFC. The presentations were 
grouped into the following panels: 

• Experimental design 

• Models, measures, and metrics 

• Data collection 

• Software engineering experiences 

The only paper related to the SEL is V. Basili and 
M. Zelkowitz (University of Maryland), "Overview of the 
Software Engineering Laboratory." 

This second workshop attempted to communicate with the 
larger software engineering research community. Approaches 
and experiences with the design of experiments and data col- 
lection were reviewed. Approximately 55 persons attended 
this meeting. NTIS 
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2.12 PROCEEDINGS FROM THE THIRD SUMMER SOFTWARE ENGINEERING 
WORKSHOP . SEL-78-005 , SEPTEMBER 1978, 132 PAGES 

This document reproduces the presentations made by partici- 
pants at the Third Summer Software Engineering Workshop held 
on September 18, 1978, at GSFC. The presentations were 
grouped into the following panels: 

• The data collection process 

• Validation of software development models 

• Measuring software development methodologies 

• Current activities and future directions 

Papers related to the SEL are 

• V. Basili and M. Zelkowitz (University of Maryland), 
"The Software Engineering Laboratory — 1978" 

• R. Reiter, Jr. (University of Maryland), "Investi- '? 

gating Software Development Approaches: A Synopsis" 

Many of the discussions at this third workshop dealt with 
"how" one collects software data and "how" one conducts suc- 
cessful software experiments. Approximately 70 persons at- 
tended this meeting. NTIS 
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2 . 13 PROCEEDINGS FROM THE FOURTH SUMMER SOFTWARE ENGINEERING 
WORKSHOP . SEL-79-005, NOVEMBER 1979, 282 PAGES 

This document reproduces the presentations made by partici- 
pants at the Fourth Summer Software Engineering Workshop 
held on November 19, 1979, at GSFC. The presentations were 
grouped into the following panels: 

• The SEL 

• Data collection 

• Experiments in methodology evaluation 

• Software resource models 

• Models and metrics of software development 
Papers related to the SEL are 

• P. Belford (CSC), "Central Flow Control Software 

Development: A Case Study of the Effectiveness of 

Software Engineering Techniques" 

• V. Basili (University of* Maryland) , "Investigations 
Into Software Development in the Software Engineer- 
ing Laboratory" 

• V. Church (CSC), "Software Engineering Laboratory — 

The Data Collection Process" 

• F. McGarry (NASA/GSFC) , "Overview of the Software 
Engineering Laboratory" 

• M. Zelkowitz (University of Maryland) , "Software 

Engineering Laboratory: Data Validation" 

This fourth workshop focused on actual experiences of data 
collection and the application of software methodologies, 
models, and tools. Approximately 100 persons attended this 
meeting. NTIS 
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2.14 PROCEEDINGS FROM THE FIFTH ANNUAL SOFTWARE ENGINEERING 
WORKSHOP . SEL-80-006, NOVEMBER 1980, 242 PAGES 

This document reproduces the presentations made by partici- 
pants at the Fifth Annual Software Engineering Workshop held 
on November 24, 1980, at GSFC. The presentations were 
grouped into the following panels: 

• The SEL 

• Software cost/resource modeling 

• Software reliability 

• Measurement of the development process 
Papers related to the SEL are 

• V. Basili (University of Maryland), "Measuring the 
Effects of Specific Software Methodologies Within 
the SEL" 

• F. McGarry (NASA/GSFC) , "An Approach To Measuring 
Software Technology" 

• J. Page (CSC), "Impacts of Experiments and Software 
Technology Changes in a Production Environment" 

This fifth workshop focused on actual experiences with the 
application of software methodologies and models. Approxi- 
mately 140 persons attended this meeting. NTIS 
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2.15 PROCEEDINGS OF THE SIXTH ANNUAL SOFTWARE ENGINEERING 
WORKSHOP . SEL-81-013 , DECEMBER 1981, 282 PAGES 

This document reproduces the presentations made by partici- 
pants at the Sixth Annual Software Engineering Workshop held 
on December 2, 1981, at GSFC. The presentations were 
grouped into the following panels: 

• Evaluating software development characteristics 

• Software metrics 

0 Software models 

• Software methodologies 

Papers related to the SEL are 

• V. Basili (University of Maryland), "Assessment of 
Software Measures in the Software Engineering Labo- 
ratory" 

• D. Card (CSC), "Identification and Evaluation of 
Software Metrics" 

• G. Page (CSC), "Evaluating the Effects of an Inde- 
pendent Verification and Validation Team" 

The document also includes a summary of the presentations 
and audience comments. This sixth workshop was an attempt 
to gather the experiences of software developers in applying 
modern programming practices and other software engineering 
techniques. Approximately 200 persons attended this meet- 
ing. NTIS 
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2.16 PROCEEDINGS OF THE SEVENTH ANNUAL SOFTWARE ENGINEERING 
WORKSHOP . SEL-82— 007, DECEMBER 1982, 400 PAGES 

This document reproduces the presentations made by partici- 
pants at the Seventh Annual Software Engineering Workshop 
held on December 1, 1982, at GSFC. The presentations were 
grouped into the following panels: 

• The SEL 

• Software tools 

• Software errors 

• Software cost estimation 

Papers related to the SEL are 

• V. Basili (University of Maryland), "Software Errors 
and Complexity, An Empirical Investigation" 

• I. Miyamoto (University of Maryland), "Technology 
Transfer Software Engineering Tools" 

• M. Zelkowitz (University of Maryland), "Software 

Prototyping in the Software Engineering Laboratory" 

«. 

The document also includes a summary of the presentations 
and audience remarks. The major emphasis of the meeting was' 
on reporting and discussing actual experiences with software 
methodologies, models, and tools. Approximately 250 people, 
representing 9 universities, 22 agencies of the Federal 
Government, and 43 private organizations, attended the meet- 
ing. NTIS 
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2.17 PROCEEDINGS OF THE EIGHTH ANNUAL SOFTWARE ENGINEERING 
WORKSHOP . SEL-83— 007, NOVEMBER 1983, 316 PAGES 

This document reproduces the presentations made by partici- 
pants at the Eighth Annual Software Engineering Workshop 
held on November 3, 1983 at GSFC. The presentations were 
grouped into the following panels: 

• The SEL 

• Software testing 

• Human factors in software engineering 

• Software quality assessment 

Papers related to the SEL are 

• V. Basili and C. Doerflinger (University of 
Maryland) , "Monitoring Software Development Through 
Dynamic Variables" 

• D. Card (CSC), "Evaluating Software Engineering 

Technologies in the SEL" 

• C. Grantham (University of Maryland), "Evaluating 
Multiple Coordinated Windows for Programmer Work- 
stations" 

• J. Ramsey (University of Maryland), "Structural 

Coverage of Functional Testing" 

• M. Zelkowitz (University of Maryland), "Character- 
istics of a Prototyping Experiment" 

The document also includes a summary of the presentations 
and audience remarks. Approximately 250 people representing 
5 universities, 23 agencies of the Federal Government, and 
44 private companies attended the meeting. NTIS 
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2.18 PROCEEDINGS OF THE NINTH ANNUAL SOFTWARE ENGINEERING 
WORKSHOP . SEL-84— 004 , NOVEMBER 1984, 349 PAGES 

This document reproduces the presentations made by partici- 
pants at the Eighth Annual Software Engineering Workshop 
held on November 28, 1984, at GSFC. The presentations were 
grouped into the following panels: 

• The SEL 

• Software error studies 

• Experiments with software development 

• Software tools 

Papers related to the SEL are 

• W. Agresti (CSC), "An Approach to Developing 
Specification Measures" 

o R. Basili, R. Selby, et al., (University of 

Maryland), "Evaluating Software Testing Strategies" 

• V. Basili et al., (University of Maryland) 

"Software Development in Ada" 

• H. Rombach (University of Maryland), "Design Metrics 

for Maintenance" 

The document also includes a summary of the presentations 
and audience remarks. Approximately 300 people, repre- 
senting 7 universities, 26 agencies of the Federal Govern- 
ment, and 56 private companies, attended the meeting. NTIS 
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2.19 PROCEEDINGS OF THE TENTH ANNUAL SOFTWARE ENGINEERING 
WORKSHOP . SEL-85— 006, DECEMBER 1985, 360 PAGES 

This document reproduces the presentations made by partici- 
pants at the Eighth Annual Software Engineering Workshop 
held on December 4, 1985, at GSFC. The presentations were 
grouped into the following panels: 

• The SEL 

• Tools for software management 

•• Software environments 

• Experiments with Ada 

Papers related to the SEL are 

• B. Agresti (CSC), "Measuring Ada As a Software 
Development Technology in the SEL" 

• V. Basili (University of Maryland), "Can We Measure 
Software Technology; Lessons From 8 Years of Trying" 

• F. E. McGarry (NASA/GSFC) , "Recent SEL Studies" 

• J. Valett (NASA/GSFC) and A. Raskin (Yale), 

"DEASEL: An Expert System for Software Engineering" 

The document also includes a summary of the presentations 
and audience remarks. Approximately 400 people, repre- 
senting 6 universities, 27 agencies of the Federal Govern- 
ment, and 55 private companies, attended the meeting. NTIS 
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2.20 PROCEEDINGS OF THE ELEVENTH ANNUAL SOFTWARE ENGINEER- 
ING WORKSHOP . SEL-86— 006, DECEMBER 1986, 308 PAGES 

This document reproduces the presentations made by partici- 
pants at the Eleventh Annual Software Engineering Workshop 
held on December 3, 1987, at GSFC. The presentations were 
grouped into the following panels: 

• The SEL 

• Empirical studies of software technology 

• Software environments 

• Software testing 

Papers related to the SEL are 

• F. McGarry, S. Voltz, and J. Valett (NASA/GSFC), 
"Determining Software Productivity Leverage Factors 
in the SEL" 

• V. R. Basili and H. D. Rombach (University of 

Maryland), "TAME: Tailoring A Measurement Environ- 

ment " 

• M. V. Zelkowitz (University of Maryland), "Automat- 
ing the Design Process with Syntactic-Based Tools" 

• W. W. Agresti (CSC), "SEL Ada Experiment: Status 

and Design Experiences" 

The document also includes a summary of the presentations 
and audience remarks. Approximately 360 people, represent- 
ing 59 private corporations, 9 universities, 18 agencies of 
the Federal Government, and 8 NASA centers, attended the 
meeting. NTIS 
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2.21 PROCEEDINGS OF THE TWELFTH ANNUAL SOFTWARE ENGINEER- 
ING WORKSHOP . SEL-87-010, DECEMBER 1987, 479 PAGES 

This document reproduces the presentations made by partic- 
ipants at the Twelfth Annual Software Engineering Workshop 
held on December 2, 1987, at GSFC. The presentations were 
grouped into the following panels: 

• Studies and experiments with Ada 

• Experiments with environments 

• Case studies 

• Measures of cost and reliability 

The only paper related to the SEL is 

• S. Godfrey (NASA/GSFC) and C. Brophy (University of 
Maryland) , "An Experiment in Ada in the Software 
Engineering Laboratory - Lessons Learned from the 
Ada Code/Unit Test Phase" 

The document also includes a summary of the presentations 
and audience remarks. Approximately 400 people, represent- 
ing 59 private corporations, 10 universities, 14 agencies of 
the Federal Government, and 7 NASA centers attended the 
workshop. NTIS 


2.22 PROCEEDINGS OF THE THIRTEENTH ANNUAL SOFTWARE ENGI- 
NEERING WORKSHOP . SEL-88— 004, NOVEMBER 1988, 379 PAGES 

This document reproduces the presentations made by partici- 
pants at the Thirteenth Annual Software Engineering Work- 
shop, held on November 30, 1988, at GSFC. The presentations 
were grouped into the following panels: 

• Studies and experiments in the SEL 

• Software models 

• Study of software products 

• Tools 

Papers related to the SEL are 

• F. McGarry (NASA/GSFC), L. Esker, and K. Quimby 
(CSC) , "Evolving Impact of Ada on a Production 
Software Environment" 

• V. R. Basili and H. D. Rombach (University of 

Maryland), "Towards a Comprehensive Framework for.. . 
Reuse: A Reuse-Enabling Software Evolution 

Environment" 

• J. D. Valett (NASA/GSFC), W. Decker, and J. Buell- 
(CSC), "The Software Management Environment" 

The document also includes a summary of the presentations 
and audience remarks. Approximately 450 people, represent- 
ing 78 private corporations, 6 universities, 22 agencies of 
the Federal Government, and 8 NASA centers, attended the 
workshop. NTIS 
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2.23 PROCEEDINGS OF THE FOURTEENTH ANNUAL SOFTWARE ENGI- 
NEERING WORKSHOP . SEL-89— 007, NOVEMBER 1989, 390 PAGES 


This document reproduces the presentations made by partici- 
pants at the Fourteenth Annual Software Engineering Work- 
shop, held on November 29, 1989, at GSFC. The presentations 
were grouped into the following panels: 

• Studies and Experiments in the SEL 

• Methodologies 

• Software Reuse * 

• Testing and Error Analysis 

Papers related to the SEL are 

• V. R. Basili (University of Maryland), "The 

Experience Factory: Packaging Software Experience" 

• F. McGarry (NASA/GSFC) , S. Waligora and 

T. McDermott (CSC), "Experiences in the Software 
Engineering Laboratory (SEL) Applying Software 
Measurement" 

• A. Kouchakdjian (University of Maryland), S. Green 

(NASA/GSFC), and V. Basili (University of Maryland), 
"Evaluation of the Cleanroom Methodology in the 
Software Engineering Laboratory" 

• H. D. Rombach and B. T. Ulery (University of 
Maryland), J. Valett (NASA/GSFC), "Measurement 
Based Improvement of Maintenance in the SEL" 

The document also includes a summary of the presentations 
and audience remarks. Approximately 450 people, representing 
83 private corporations, 10 universities, 22 agencies of the 
Federal Government, and 8 NASA centers, attended the work- 
shop. NT IS 
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2.24 PROCEEDINGS OF THE FIFTEENTH ANNUAL SOFTWARE 

ENGINEERING WORKSHOP . SEL-90-006, NOVEMBER 1990, 

566 PAGES 

This document reproduces the presentations made by 
participants at the Fourteenth Annual Software Engineering 
Workshop, held on November 28 and 29, 1990, at GSFC. The 
presentations were grouped into the following sessions: 

• The SEL- at Age- 15 

• Process Improvement 

• Measurement 

• Reuse 

• Process Assessment 

The sessions were followed by two panel discussions: 

• Experiences in Implementing an Effective 
Measurement Program 

• Software Engineering in the 1980s: Most Significant 

Achievements/Greatest Disappointments 

Papers related to the SEL are 

• V. R. Basili (University of Maryland), "Towards a 

Mature Measurement Environment: Creating a 

Software Engineering Research Environment" 

• G. Heller and G. Page (CSC), "Impact of a Process 

Improvement Program in a Production Software 
Environment: Are We Any Better?" 

• F. McGarry and R. Pajerski (NASA/GSFC), "Towards 
Understanding Software — 15 Years in the SEL" 

• P. A. Straub and M. V. Zelkowitz (University of 
Maryland), "Bias and Design in Software 
Specifications" 

• R. Kester (CSC), "SEL Ada Reuse Analysis and 
Representations" 
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The document also includes a summary of the presentations 
and audience remarks. Approximately 500 people, 
representing 100 private corporations, 10 universities, 

30 agencies of the Federal Government, and 9 NASA centers, 
attended the workshop. NTIS 
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2.25 "THE SOFTWARE ENGINEERING LABORATORY: OBJECTIVES , " 

V. R. BASILI AND M. V. ZELKOWITZ, PROCEEDINGS OF 
THE FIFTEENTH ANNUAL CONFERENCE ON COMPUTER PERSONNEL 
RESEARCH . AUGUST 1977, 14 PAGES 

This technical paper provides an overview of the SEL and its 
objectives. The original motivations for establishing the 
SEL were the high cost of software development and the sub- 
sequent need to optimize the- development process. This 
paper discusses the following aspects of the SEL with re- 
spect to these motivations: 

• Specific objectives of the SEL 

• Software development factors to be investigated 

• Data collection techniques 

• Early SEL research activities 

The importance of defining consistent software development 
measures is a recurrent theme throughout the discussion. JAO 


This technical paper also appears in SEL-82-00.4, Collected 
Software Engineering Papers: Volume I . July 1982. 
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2.26 "OPERATION OF THE SOFTWARE ENGINEERING LABORATORY," 

V. R. BASIL I AND M. V. ZELKOWITZ, PROCEEDINGS OF THE 
SECOND SOFTWARE LIFE CYCLE MANAGEMENT WORKSHOP . 

AUGUST 1978, 4 PAGES 

This technical paper describes the operation of the SEL. 
Software engineering data is regularly collected by the SEL 
from flight dynamics software development projects at GSFC. 
The assembled data supports an extensive program of software 
engineering research. This report also reviews SEL data 
collection and data processing activities • and their rela- 
tionship to the research program. It also summarizes some 
ongoing resource estimation and error analysis research 
projects. JAO 


This technical paper also appears in SEL-82-004, Collected 
Software Engineering Papers: Volume I . July 1982. 
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2.27 THE SOFTWARE ENGINEERING LABORATORY . SEL-81-104, 

D. N. CARD, F. E. McGARRY, G. PAGE, ET AL., 

FEBRUARY 1982, 121 PAGES 

This document describes the history, organization, opera- 
tion, and research results of the SEL. The SEL is a joint 
effort of GSFC , Computer Sciences Corporation, and the 
University of Maryland. The objective of the SEL is to 
study and improve the software development process in the 
GSFC environment. The SEL has conducted extensive research 
in the following areas of software engineering: 

• Methodology evaluation 

• Tool evaluation 

• Resource models 

• Reliability models 

• Software measures 

The document outlines SEL efforts in these areas and pre- 
sents some preliminary conclusions based on this work. The 
appendixes include descriptions of the software projects 
studied and summary statistics derived from this data. This 
document was also issued as CSC/TM-82/6033 . NTIS 


The previous version of this document was The Software Engi- 
neering Laboratory . SEL-81-004, D. N. Card, F. E. McGarry, 

G. Page, et al., September 1981. 
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2.28 GLOSSARY OF SOFTWARE ENGINEERING LABORATORY TERMS . 

SEL-82-105 , T. A. BABST, M. G. ROHLEDER, AND 

F. E. McGARRY, NOVEMBER 1983, 39 PAGES 

This document is a glossary of terms used in the SEL. A 
list of acronyms is also included. The terms are defined 
within the context of the software development environment 
for flight dynamics at GSFC. The purposes of this document 
are to provide a concise reference for clarifying the lan- 
guage employed in SEL documents and data collection forms, 
establish standard definitions for use by SEL personnel, and 
explain basic software engineering concepts. A version of 
this document was also issued as Computer Sciences Corpora- 
tion document CSC/TM-83/6168 . SDB 


The previous version of this document was Glossary of 
Software Engineering Laboratory Terms . SEL-82-005, 

M. G. Rohleder, December 1982. 
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2.29 


SOFTWARE ENGINEERING LABORATORY ( SEL) DATA AND 
INFORMATION POLICY (REVISION 1 ) . SEL-91-102, 

F. MCGARRY, AUGUST 1991; 24 PAGES 

This document presents the policies and overall procedures 
that are used in distributing and in making available 
products of the Software Engineering Laboratory (SEL) . The 
products include project data and measures, source code, 
reports, and software tools. NTIS 


The previous version of this document was Software 
Engineering Laboratory (SEL) Data and Information Policy . 
SEL-9 1-002 , F. McGarry, April 1991. 
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SECTION 3 - THE SOFTWARE ENGINEERING LABORATORY: 
SOFTWARE DEVELOPMENT DOCUMENTS 
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3.1 RECOMMENDED APPROACH TO SOFTWARE DEVELOPMENT . 

SEL-81-205, F. E. McGARRY, G. PAGE, S'. ESLINGER, 

ET AL., APRIL 1983, 278 PAGES 

This document presents recommendations for a disciplined 
approach to software development, based on data collected 
and studied by the SEL since 1977 for approximately 
40 flight dynamics software projects. It describes the 
major activities, end products, methodologies, tools, 
models, and measures applicable to each phase of the soft- 
ware life cycle. Life cycle phases ioclude requirements 
analysis, preliminary design, detailed design, implementa- 
tion, system testing, and acceptance testing. 

The emphasis of the document is on management considera- 
tions, including the activities specific to each phase of 
the life cycle and the general management concerns tran- 
scending phase boundaries, such as development planning and 
performance monitoring. Specific recommendations are made 
with respect to determining project quality. 

This version of the document contains new appendixes cover- 
ing the topics of software reviews, development documents, 
examples of steps to organize a project, and a summary of 
key information. This document was also issued as Computer 
Sciences Corporation document CSC/TM-83/6019 . NTIS 


The previous versions of this document were Standard Ap- 
proach to Software Development . SEL-81-005, V. E. Church, 

F. E. McGarry, and G. Page, September 1981, and Recommended 
Approach to Software Development . SEL-81-105, S. Eslinger, 
F. E. McGarry, and G. Page, May 1982. 
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3.2 MANAGER'S HANDBOOK FOR SOFTWARE DEVELOPMENT 

(REVISION 1) . SEL-84-101, L. LANDIS, F. E. McGARRY, 

S. WALIGORA ET AL., NOVEMBER 1990, 91 PAGES 

This document presents methods and aids for the management 
of software development projects. The recommendations are 
based on analyses and experiences of the SEL with flight 
dynamics software development. The management aspects of 
the following subjects are described: 

• Organizing the project 

• Producing a development plan 

• Estimating costs 

• Scheduling 

• Staffing 

• Preparing deliverable documents 

• Using management tools 

• Monitoring the project ■ 

• Conducting reviews 

• Auditing 

• Testing 

• Certifying 

Revision 1 contains extensive updates, including additional 
material on management metrics and revisions to cost esti- 
mation factors, document contents, and testing procedures. 
NTIS 


The previous version of this document was Manager's Handbook 
for Software Development . SEL-84-001, W. W. Agresti, 

F. E. McGarry, D. N. Card, et al., April 1984. 
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3.3 PROGRAMMER'S HANDBOOK FOR FLIGHT DYNAMICS SOFTWARE 
DEVELOPMENT . SEL-86-001, R. WOOD and E. EDWARDS, 

MARCH 1986, 272 PAGES 

Specific procedures, standards, and styles are provided as 
recommended guidelines for programmers' use during the de- 
tailed design and implementation phases of flight dynamics 
software development. Brief descriptions of the other life- 
cycle phases are included for reference and context. NTIS 
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3.4 SOFTWARE VERIFICATION AND TESTING . SEL-85-005, 

D. N. CARD, C. ANTLE, and E. EDWARDS, DECEMBER 1985, 

64 PAGES 

General procedures for software verification and validation 
are provided as a guide for managers, programmers, and 
analysts involved in software development. The verification 
and validation procedures described are based primarily on 
testing techniques. Testing refers to the execution of all or 
part of a software system for the purpose of detecting 
errors. Planning, execution, and analysis of tests are. out- 
lined in this document. Code reading and. static analysis 
techniques for software verification are also described. NTIS 
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3.5 PRODUCT ASSURANCE POLICIES AND PROCEDURES FOR FLIGHT 
DYNAMICS SOFTWARE DEVELOPMENT . SEL-87-001, S. PERRY 
et al., MARCH 1987, 106 PAGES 

The product assurance policies and procedures necessary to 
support flight dynamics software development projects for 
Goddard Space Flight Center are presented. The quality as- 
surance and configuration management methods and tools for 
each phase of the software development life cycle are de- 
scribed, from requirements analysis through acceptance test- 
ing; maintenance and operation are not addressed. CAST 


This document supersedes Configuration Management and Con- 
trol: Policies and Procedures . SEL-84-002, Q. L. Jordan and 

E. Edwards, December 1984. 
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SECTION 4 - SOFTWARE TOOLS 
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4.1 COMMON SOFTWARE MODULE REPOSITORY (CSMR) SYSTEM DE- 
SCRIPTION AND USER’S GUIDE . SEL-79-003, 

C. E. GOOREVICH, A. L. GREEN, AND S. R. WALIGORA, 

AUGUST 1979, 156 PAGES 

This document is the system description and user's guide for 
the Common Software Module Repository (CSMR) . The CSMR pro- 
gram is a software library utility that provides interactive 
access to a data base of software modules. 

This document describes library control procedures, program 
capabilities, and operating procedures. The system design 
and individual module descriptions are also included. Var- 
ious appendixes contain the data base file formats and the 
system implementation procedures. This document was also 
issued as Computer Sciences Corporation document 
CSC/SD-79/6103 . SDB 
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4.2 COST AND RELIABILITY ESTIMATION MODELS ( CAREM) USER*S 
GUIDE . SEL-81— 008 , J. F. COOK AND E. EDWARDS, 

FEBRUARY 1981, 28 PAGES 

This document describes the operation of an interactive 
software cost and reliability modeling utility. The Cost 
and Reliability Estimation Models (CAREM) program allows the 
user to fit any of several common models to a selected sub- 
set of SEL data. The following models are available in the 
program: 

• Doty 

• GRC 

• Tecolote 

• Walston/Felix 

This document includes a brief description of each model, 
operating instructions, and sample sessions. This document 
was also issued as a Goddard Space Flight Center technical 
memorandum. SDB 
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4.3 SOFTWARE ENGINEERING LABORATORY ( SEL^ COMPENDIUM OF 
TOOLS (REVISION 1) . SEL-81-107, W. J. DECKER, 

W. A. TAYLOR, AND E. J. SMITH, FEBRUARY 1982, 76 PAGES 

This document contains a series of brief descriptions of 
software tools available on the SEL computers (a PDP-11/70 
and a VAX-11/780) . The tools described in this document 
support the following applications: 

• Cost and resource modeling 

• Configuration management 

• ' Software library management 

• Data base maintenance 

• Document library indexing 

• Financial reporting 

• Requirements analysis 

• Source code analysis 

• Structured FORTRAN 

These brief descriptions help potential users to judge the 
suitability of these programs to their needs. This document 
was also issued as Computer Sciences Corporation document 
CSC/TM-82/6038. NTIS 


The previous -version of this document was Software Engineer- 
ing Laboratory (SEL) Compendium of Tools . SEL-81-007, 

W. J. Decker, E. J. Smith, W. A. Taylor, et al., February 
1981. 
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4.4 FORTRAN STATIC SOURCE CODE ANALYZER PROGRAM ( SAP) 

SYSTEM DESCRIPTION (REVISION 1) . SEL-82-102, 

W. A. TAYLOR AND- W. J. DECKER , APRIL 1985, 217 PAGES 

This document presents the FORTRAN Static Source Code 
Analyzer Program (SAP) system description (Revision 1) . SAP 
is a software tool designed to assist SEL personnel in con- 
ducting studies of FORTRAN programs. SAP scans FORTRAN 
source code and produces reports giving statistics and 
measures of statements and structures that make up a mod- 
ule. The document describes the processing performed by 
SAP; the routines, COMMON blocks, and files used by SAP; and 
the SAP system generation procedure. 

This document follows the SAP tool specifics on the 
VAX-11/780, Version 3. The IBM 4341 is the batch Ver- 
sion 3. Departures from the VAX-11/780 version are noted. 
The PDP- 1.1/70 version is an older version and is presented 
in SEL-82-002 , FORTRAN Static Source Code Analyzer Program 
(SAP) System Description . August 1982. SDB 


The previous version of this document was FORTRAN Static 
Source Code Analyzer Program (SAP) System Description . 
SEL-82-002, W. J. Decker and W. A. Taylor, August 1982. 
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4.5 FORTRAN STATIC SOURCE CODE ANALYZER PROGRAM (SAP) 

USER'S GUIDE (REVISION 3) . SEL-78-302, W. J. DECKER 
AND W. A. TAYLOR , JULY 1986, 145 PAGES 

This document presents the FORTRAN Static Source Code Ana- 
lyzer Program (SAP) User's Guide (Revision 3). SAP is a 
software tool designed to assist SEL personnel in conducting 
studies of FORTRAN programs. SAP scans FORTRAN source code 
and produces reports that present statistics and measures of 
statements and structures that make up a module. The docu- 
ment provides instructions for operating SAP and contains 
information useful in interpreting SAP output. It is a 
revision of the previous SAP user's guide, SEL-78-202, and 
is the result of integrating SAP into the Software Develop- 
ment Environment (SDE) . CAS I 


The previous versions of this document were FORTRAN Static 
Source Code Analyzer Program (SAP) User's Guide . SEL-78-002, 
E. M. O'Neill, S. R. Waligora, C. E. Goorevich, et al., 
February 1978; FORTRAN Static Source Code Analyzer Program 
(SAP) User's Guide (Revision 1) . SEL-78-102, W. J. Decker 
and W. A. Taylor, September 1982; and FORTRAN Static Source 
Code Analyzer Program (SAP) User's Guide (Revision 2) . SEL- 
78-202, W. J. Decker and W. A. Taylor, April 1985. 
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4.6 FLIGHT DYNAMICS SYSTEM SOFTWARE DEVELOPMENT ENVIRONMENT 
(FDS/SDE) TUTORIAL . SEL-86-003, J. C. BUELL AND 
P. I. MYERS, JULY 1986, 137 PAGES 

The Flight Dynamics System Software Development Environment 
(FDS/SDE) is an interactive tool for developing software in 
the flight dynamics environment. It uses a menu-driven, 
f ill-in-the-blanks format that permits the developer to input, 
edit, compile, link, and execute software. Online help is 
provided at all steps, minimizing training time to use the 
tool. This document steps through a sample development 
scenario using the FDS/SDE, presenting sample displays and 
user responses. NTIS 
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4.7 SOFTWARE MANAGEMENT ENVIRONMENT (SME) CONCEPTS AND 
ARCHITECTURE . SEL-89-003, W. DECKER AND J. VALETT, 
AUGUST 1989, 48 PAGES 

This document presents the concepts and architecture of the 
Software Management Environment (SME) developed for the Sys- 
tems Development Branch (Code 552) of the Flight Dynamics 
Division of GSFC. The purpose of SME is to provide an inte- 
grated set of management tools that can assist software de- 
velopment managers in managing and planning flight dynamics 
software development projects. NTIS 
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5.1 APPLICABILITY OF THE RAYLEIGH CURVE TO THE SEL ENVIRON- 
MENT . SEL-78-007, T. E. MAPP, DECEMBER 1978, 27 PAGES 


This document reviews the resource utilization model for 
software development, which is based on the Rayleigh curve 
developed by Norden and Putnam. A Rayleigh curve is fit to 
data provided by the SEL. Parabolas, trapezoids, and 
straight lines are also fit to the same data. The parabola 
and trapezoid give about as good a fit as the Rayleigh curve. 
Therefore, this document concludes that while the Rayleigh 
curve may be an appropriate model for resource expenditures, 
it is not necessarily the best model for small- to medium- 
size projects. CAS I • 
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5.2 "RESOURCE ESTIMATION FOR MEDIUM-SCALE SOFTWARE PROJ- 
ECTS," M. V. ZELKOWITZ, PROCEEDINGS OF THE TWELFTH CON- 
FERENCE ON THE INTERFACE OF STATISTICS AND COMPUTER 
SCIENCE . NEW YORK: IEEE COMPUTER SOCIETY PRESS, 1979, 

6 PAGES 

This technical paper describes the analysis of resource es- 
timation techniques that is being performed by the SEL. The 
data used in the analysis is collected from medium-scale 
flight dynamics software development projects at GSFC. A 
procedure to forecast accurately the cost and development 
time of these projects would be a valuable management tool 
in this environment. This paper documents a specific at- 
tempt to verify the resource estimation model based on the 
Rayleigh curve that was developed by Norden and Putnam. JAO 


This technical paper also appears in SEL-82-004, Collected 
Software Engineering Papers: Volume I . July 1982. 
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5.3 THE SOFTWARE ENGINEERING LABORATORY: RELATIONSHIP 

EQUATIONS . SEL-79-002 , K. FREBURGER AND V. R. BASILI, 
MAY 1979 , 67 PAGES 

This document presents the results of an analysis of several 
factors affecting software development. The analysis was 
based on data collected by the SEL. Relationships among the 
following measures were studied: 

• Total effort (staff-months) 

• Lines of delivered code (thousands) 

• Lines of developed code (thousands) 

• Percentage of developed code 

• Number of modules 

• Number of developed modules 

• Percentage of developed modules 

• Project duration (months) 

• Pages of documentation 

• Productivity 

• Average staff size 

Estimating equations were derived from the measures by sta- 
tistical analysis and were then compared with results ob- 
tained by Walston and Felix in a similar study. This 
document was also issued as University of Maryland Technical 
Report TR-764 . CAST 
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5.4 TUTORIAL ON MODELS AND METRICS FOR SOFTWARE MANAGEMENT 
AND ENGINEERING . SEL-80-008, V. R. BASILI, 1980, 

349 PAGES 

This document is a tutorial on quantitative methods of soft- 
ware management and engineering. A quantitative methodology 
is needed to evaluate, control, and predict software devel- 
opment and maintenance costs. This quantitative approach 
allows cost, time, and quality tradeoffs to be made in a 
systematic manner. The tutorial focuses on numerical 
product-oriented measures such as size, complexity, and 
reliability and on resource-oriented measures such as cost, 
schedules, and resources. Twenty articles from software 
engineering literature are reprinted in this document. The 
articles are organized into the following sections: 

• Resource models 

• Changes and errors 

• Product metrics 

• Data collection 

Successful application of these techniques, however, re- 
quires a thorough knowledge of the project under development 
and any assumptions made. Only then can these techniques 
augment good managerial and engineering judgment. JAO 


This document was published as the IEEE tutorial. Models and 
Metrics for Software Management and Engineering . New York: 
Computer Societies Press, 1980. 
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5.5 "MODELS AND METRICS FOR SOFTWARE MANAGEMENT AND ENGI- 
NEERING," V. R. BASILI, ASME ADVANCES IN COMPUTER 
TECHNOLOGY . JANUARY 1980, VOL. 1, 12 PAGES 

This technical paper attempts to characterize several quan 
titative models and measures of the software development 
process. These models and measures deal with various as- 
pects of the software process and product, including re- 
source estimation, complexity, reliability, and size. The 
relationship of these models and measures to the software 
development life cycle is also discussed. Finally, the ex 
tent to which the various models have been applied in pro- 
duction environments and the success they have achieved is 
indicated. JAO 


This technical paper also appears in SEL-82-004, Collected 
Software Engineering Papers: Volume I . July 1982. 
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5.6 A STUDY OF THE MUSA RELIABILITY MODEL . SEL-80-005, 

A. M. MILLER, NOVEMBER 1980, 94 PAGES 

This document describes a study in which the Musa reliabil- 
ity model was applied to three software projects developed 
for GSFC, with the goal of determining whether the model 
could be used in the flight dynamics environment as a soft- 
ware management tool. One purpose of the model is to pre- 
dict the total number of errors in a piece of software 
undergoing testing. Actual times between failures and their 
associated run times were fitted to the Musa equation in an 
iterative procedure. Of the three projects studied, the 
results for one converged to a value 25 percent higher than 
the actual number of errors; the other two did not converge 
at all. 

The document discusses the assumptions underlying the model 
and evaluates the characteristics of the environment that 
could affect these assumptions. Suggestions are offered 
about changes that could be made in the environment to 
better meet the assumptions. This document was originally 
prepared as a Master's Thesis at the University of Maryland. 
CAST 
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5.7 AN APPRAISAL OF SELECTED COST/RESOURCE ESTIMATION 

MODELS FOR SOFTWARE SYSTEMS . SEL-80-007, J. F. COOK 
AND F. E. McGARRY, DECEMBER 1980, 41 PAGES 

This document presents the results of an evaluation and com- 
parison of seven cost/resource estimation models based on 
SEL data. The following models were considered: 

• Doty 

• Walston/Felix 

• Tecolote 

• GRC 

• SLIM 

• PRICE S3 

• SEL Meta-Model 

The validity of the theoretical bases of these models was 
not analyzed. The objective of the appraisal was simply to 
determine how well SEL data conformed to the predictions of 
various models. This document was also issued as Goddard 
Space Flight Center document X-582-81-1. NTIS 

\ 
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5.8 "A META-MODEL FOR SOFTWARE DEVELOPMENT RESOURCE 
EXPENDITURES," J. W. BAILEY AND V. R. BASILI, 
PROCEEDINGS OF THE FIFTH INTERNATIONAL CONFERENCE ON 
SOFTWARE ENGINEERING . NEW YORK: IEEE COMPUTER 

SOCIETY PRESS, 1981, 10 PAGES 

This technical paper describes an effort to produce a model 
of software development resource expenditures that can be 
generalized to a number of situations. Many models have 
been proposed over the last several years. However, experi- 
ence has shown that differences in the data collected, types 
of projects developed, and environmental factors limit the 
transportability of these models from one organization to 
another. This conclusion is reasonable because a model de- 
veloped in any given environment will reflect only the im- 
pact of factors that have a variable effect in that 
environment. Factors that are constant in that environment 
(and therefore do not affect productivity) may have differ- 
ent or variable effects in another environment. 

This paper describes a model-generation process that permits 
the development of a resource estimation model for any par- 
ticular organization. The process provides the capability 
to produce a model that is tailored to the organization and 
can be expected to be more effective than any model orig- 
inally developed for another environment. The model is 
demonstrated here using data collected by the SEL at GSFC. 
JAO 


This technical paper also appears in SEL-82-004, Collected 
Software Engineering Papers: Volume I . July 1982. 
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5.9 


"CAN THE PARR CURVE HELP WITH MANPOWER DISTRIBUTION AND 
RESOURCE ESTIMATION PROBLEMS?," V. R. BASILI AND 
J. BEANE, JOURNAL OF SYSTEMS AND SOFTWARE . 

FEBRUARY 1981, VOL. 2, NO. 1, 11 PAGES 

This technical paper analyzes the resource utilization model 
developed by Parr. The curve predicted by the model is com- 
pared with several other curves, including the Rayleigh 
curve, a parabola, and a trapezoid, with respect to how well 
they fit manpower utilization. The evaluation is performed 
for several flight dynamics projects of the 6- to 12-man-year 
effort range that were studied by the SEL. 

The conclusion drawn is that the Parr curve can be made to 
fit the data better than the other curves. However, because 
of the noise in the data, it is difficult to confirm the 
shape of the manpower distribution from the data alone and 
therefore difficult to validate any particular model. More- 
over, since the parameters used in the curve are not easily 
calculable or estimable from known data, the curve is not 
effective for resource estimation. JAO 


This technical paper also appears in SEL-82-004, Collected 
Software Engineering Papers: Volume I . July 1982. 
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5.10 THE RAYLEIGH CURVE AS A MODEL FOR EFFORT DISTRIBUTION 

OVER THE LIFE OF MEDIUM SCALE SOFTWARE SYSTEMS . 

SEL-81-012 , G. O. PICASSO, DECEMBER 1981, 153 PAGES 

This document discusses some of the factors affecting the 
accuracy of resource models applied to medium-scale software 
systems. Putnam has shown that the Rayleigh curve is an 
adequate model for the life-cycle effort distribution of 
large-scale systems. Previous investigations of the appli- 
cability of this model to medium-scale software development 
efforts have met with mixed results. The results of the 
earlier investigations are confirmed in this analysis. .The 
reasons for the failure of the models are found in the sub- 
cycle (phase) effort data. There are four contributing fac- 
tors: uniqueness of the environment studied, the influence 

of holidays, varying management techniques, and differences 
in the data studied. This document was also issued as 
University of Maryland Technical Report TR-1186. SDB 
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5.11 "COMPARISON OF REGRESSION MODELING TECHNIQUES FOR 

RESOURCE ESTIMATION, “ D. N. CARD, COMPUTER SCIENCES 
CORPORATION, TECHNICAL MEMORANDUM, NOVEMBER 1982, 

21 PAGES 

This technical memorandum presents the results of a study- 
conducted to compare three alternative regression procedures 
by examining the results of their application to one com- 
monly accepted equation for resource estimation. Linear, 
Log-Linear, and Nonlinear procedures were considered. The 
memorandum summarizes the data studied, describes the re- 
source estimation equation, explains the regression proce- 
dures, and compares the results obtained from the 
procedures. The regression procedures were evaluated with 
respect to numerical accuracy, conceptual accuracy, and 
computational cost. This study is based on data collected 
from 22 flight dynamics software projects studied by the 
SEL. SDB 


This technical paper also appears in SEL-83-003, Collected 
Software Engineering Papers: Volume II . November 1983. 
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5.12 "MONITORING SOFTWARE DEVELOPMENT THROUGH DYNAMIC 
VARIABLES," C. W. DOERFLINGER AND V. R. BASILI, 
PROCEEDINGS OF THE SEVENTH INTERNATIONAL COMPUTER 
SOFTWARE AND APPLICATIONS CONFERENCE . NEW YORK: 

IEEE COMPUTER SOCIETY PRESS, 1983, 27 PAGES 

This paper summarizes the SEL document (SEL-83-006) of the 
same name. It describes research conducted by the SEL on 
the use of dynamic variables as a tool for monitoring soft- 
ware development. The intent of the project, which examined 
several FORTRAN projects with similar profiles, was to iden- 
tify project-independent measures. The. projects developed 
serve similar functions, and because the projects are simi- 
lar, some underlying relationships exist that are invariant 
between the projects. These relationships, once well de- 
fined, may be used to compare the development of different 
projects to determine whether they are evolving in the same 
way previous projects in this environment evolved. JAO 


This technical paper also appears in SEL-83-003, Collected 
Software Engineering Papers: Volume II . November 1983. 
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5.13 MONITORING SOFTWARE DEVELOPMENT THROUGH DYNAMIC VARI- 
ABLES . SEL-83-006, C. W. DOERFLINGER, NOVEMBER 1983, 
110 PAGES 

This document describes research conducted by the SEL on the 
use of dynamic variables as a tool for monitoring software 
development. The intent of the project, which examined sev- 
eral FORTRAN projects with similar profiles, was to identify 
project-independent measures. The projects developed serve 
similar functions, and because the projects are similar, 
some underlying relationships exist that are invariant be- 
tween the projects. These relationships, once well defined, 
may be used to compare the development of different projects 
to determine whether they are evolving in the same way pre- 
vious projects in this environment evolved. This document 
was originally prepared as a Master's Thesis at the 
University of Maryland. SDB 
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5.14 AN APPROACH TO SOFTWARE COST ESTIMATION . SEL-83-001, 

F. E. McGARRY, G. PAGE, D. N. CARD, ET AL., 

FEBRUARY 1984, 73 PAGES 

This document describes the general procedures for software 
cost estimation in any environment. First, the basic con- 
cepts of work and effort estimation are explained, some pop- 
ular resource estimation models are reviewed, and the 
accuracy of resource estimates is investigated. Next, gen- 
eral guidelines are presented for cost estimation throughout 
the software life cycle. The sources of information and 
relevant parameters available during each phase cycle are 
identified. Finally, a comprehensive software cost predic- 
tion procedure based on the experiences of the SEL in the 
flight dynamics area and incorporating management expertise, 
cost models, and historical data is provided. The methodol- 
ogy developed incorporates these elements into a customized 
management tool for software cost prediction. This document 
was also issued as Computer Sciences Corporation document 
CSC/TM-83/6076 . NTIS 
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"FINDING RELATIONSHIPS BETWEEN EFFORT AND OTHER 
VARIABLES IN THE SEL," V. R. BASILI AND 
N. M. PANLILIO-YAP, PROCEEDINGS OF THE NINTH 
INTERNATIONAL COMPUTER SOFTWARE AND APPLICATIONS 
CONFERENCE . NEW YORK: IEEE COMPUTER SOCIETY PRESS, 

1985, 7 PAGES 

This study examines the relationship between effort and other 
variables for 23 SEL projects that were developed for NASA/ 
GSFC . These variables fell into two categories: those that 
can be determined in the early stages of project development 
and may therefore be useful in a baseline equation for pre- 
dicting effort in future projects, and those that can be used 
mainly to characterize or evaluate effort requirements and 
thus enhance our understanding of the software development 
process in this environment. Some results of the analyses 
are presented in this paper. JAO 


This technical paper also appears in SEL-85-003, Collected 
Software Engineering Papers: Volume III . November 1985, and 

in University of Maryland Technical Report, TR-1520, July 
1985 . 
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5.16 "EXPERIMENTATION IN SOFTWARE ENGINEERING," 

V. R. BASIL I , R. W. SELBY, JR., AND D. H. HUTCHENS, 
TREE TRANSACTIONS ON SOFTWARE ENGINEERING . JULY 
1986, 11 PAGES 

This paper presents a framework for analyzing most of the 
experimental work performed in software engineering over the 
past several years. The framework of experimentation con- 
sists of four phases of the experimentation process. These 
are 

• Definition — Motivation, object, purpose, 
perspective, domain, and scope 

• Planning — Design, criteria, and measurement 

• Operation — Preparation, execution, and analysis 

• Interpretation — Interpretation context, 
extrapolation, and impact 

The paper describes a variety of experiments in the frame- 
work and discusses their contribution to the software engi- 
neering discipline. Some useful recommendations for the 
application of the experimental process in software engi- 
neering are included. JAO 


This technical paper also appears in SEL-86-004, Collected 
Software Engineering Papers: Volume IV . November 1986. 
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5.17 A STUDY ON FAULT PREDICTION AND RELIABILITY ASSESSMENT 
IN THE SEL ENVIRONMENT . TR-1699, V. R. BASILI AND 
D. PATNAIK, UNIVERSITY OF MARYLAND, TECHNICAL REPORT, 
AUGUST 1986 

This technical report presents an empirical study on fault 
estimation and prediction, prediction of fault detection and 
correction effort, and reliability assessment in the SEL 
environment. Fault estimation using empirical relationships 
and fault prediction using curve-fitting methods are inves- 
tigated. Relationships between debugging efforts (fault 
detection and correction effort) in different test phases 
are provided, to make an early estimate of future debugging 
effort. The report concludes with the fault analysis, 
application of a reliability model, and analysis of a nor- 
malized metric for reliability assessment and monitoring 
during software development. SDB 


This technical paper also appears in SEL-86-004, Collected 
Software Engineering Papers: Volume IV , November 1986. 
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5.18 


"RESOLVING THE SOFTWARE SCIENCE ANOMALY," D. N. CARD 
AND W. W. AGRESTI, THE JOURNAL OF SYSTEMS AND SOFTWARE 
1987, 6 PAGES 


This study reexamined one basic relationship proposed by the 
Halstead theory, which appears to provide a comprehensive 
model of the program construction process: that between 

estimated and actual program length. The results show that 
the apparent agreement between these quantities is a math- 
ematic artifact. Analyses of both Halstead's own data and 
another larger data set confirm this conclusion. Software 
science has neither a firm theoretical nor empirical foun- 
dation. JAO 


This technical paper also appears in SEL-87-009, Collected 
Software Engineering Papers: Volume V . November 1987. 
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5.19 


"TAILORING THE SOFTWARE PROCESS TO PROJECT GOALS AND 
ENVIRONMENTS," V. R. BASILI AND H. D. ROMBACH, PROCEED- 
INGS OF THE 9TH INTERNATIONAL CONFERENCE ON SOFTWARE 
ENGINEERING . MARCH 1987, 12 PAGES 

This paper presents a methodology for improving the software 
process by tailoring it to the specific project goals and 
environment. This improvement process is aimed at the 
global software process model as well as methods and tools 
supporting that model. The basic idea is to use defect pro- 
files to help characterize the environment and evaluate the 
project goals and the effectiveness of methods and tools in 
a quantitative way. The improvement process is implemented 
iteratively by setting project improvement goals, character- 
izing those goals and the environment, in part, via defect 
profiles in a quantitative way, choosing methods and tools 
fitting those characteristics, evaluating the actual be- 
havior of the chosen set of methods and tools, and refining 
the project goals based on the evaluation results. All 
these activities require analysis of large amounts of data 
and, therefore, support by an automated tool. Such a tool — 
TAME (Tailoring A Measurement Environment) — is currently 
being developed. JAO 


This technical paper also appears in SEL-87-009, Collected 
Software Engineering Papers: Volume V . November 1987. 
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5.20 GUIDELINES FOR APPLYING THE COMPOSITE SPECIFICATION 
MODEL fCSMl . SEL-87-003, W. W. AGRESTI , JUNE 1987, 

37 PAGES 

This document provides guidelines for applying the Composite 
Specification Model (CSM), an approach to representing soft- 
ware requirements, and for developing each of the three de- 
scriptive views of the software: 

• The contextual view, using entities and relation- 
ships 

• The dynamic view, using states and transitions 

• The functional view, using data flows and processes 

Using CSM results in a software specification document, 
which is outlined in this document. CAS I 
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5.21 A META INFORMATION BASE FOR SOFTWARE ENGINEERING . 

L. MARK AND H. D. ROMBACH, TR-1765, UNIVERSITY OF 

MARYLAND, TECHNICAL REPORT, JULY 1987, 34 PAGES 

This paper proposes a meta model and a graphical notation 
for specifying software engineering processes and products. 
This meta model leads to the view that a software engineer- 
ing information base needs to support the storage and re- 
trieval of process and product descriptions as well as all 
data related to the executions of process descriptions and 
the actual instances of product descriptions generated dur- 
ing the course of a software engineering project. A meta 
schema for information bases is presented that allows the 
authors to deal with this type of information in a natural 
way. In addition, software engineering information bases 
need to be adaptable to changing process and product de- 
scriptions based on changing project goals and character- 
istics of the project environment and the organization. The 
meta schema of an information base allows for the generation 
of a customized information base for a given set of processes 
and products specified according to the software engineering 
meta model. The idea for this research originated in the 
TAME project at the University of Maryland aiming at the 
development of a measurement, feed-back, and planning envi- 
ronment. Currently, the authors have implemented a first 
prototype information base as part of the prototype TAME 
system customized to the specific needs of the NASA/SEL en- 
vironment. The schema of this first prototype was defined 
by hand and is implemented on a relational data base system. 
Developing the idealized information base for software engi- 
neering requires more research in the areas of software 
engineering and data bases. In the area of software engi- 
neering, the authors need to improve their understanding of 
the software process and product in order to be able to 
construct more formal specifications; in the area of data 
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bases, they need to develop a data base technology for prop 
erly mirroring the specific engineering concepts, including 
self-adaptabi lity . SDB 


This paper also appears in SEL-87-009, Collected Software 
Engineering Papers: Volume V . November 1987. 
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5.22 "GENERATING CUSTOMIZED SOFTWARE ENGINEERING INFORMA- 
TION BASES FROM SOFTWARE PROCESS AND PRODUCT SPECI- 
FICATIONS," L. MARK AND H. D. ROMBACH, PROCEEDINGS OF 
THE 2 2ND ANNUAL HAWAII INTERNATIONAL CONFERENCE ON 
SYSTEM SCIENCES . JANUARY 1989, 9 PAGES 

This paper presents the information base oriented part of 
the "Meta Information Base for Software Engineering" project 
at the University of Maryland. The idea of this project is 
to generate customized software engineering information 
bases from formal specifications of software engineering 
processes and products. The generator approach acknowledges 
the fact that software engineering changes not only from 
environment to environment, but also from project to project. 
If an information base is expected to truly mirror and sup- 
port a given software engineering project, it needs to be 
tailorable to the changing characteristics of the software 
project itself. The generator bases approach suggested by 
this project seems to be the natural approach to satisfy 
this important need. This paper discusses how to represent 
a set of software process and product type specifications in 
a database and how to use these to automatically generate 
database support for process executions and product in- 
stances. JAO 


This technical paper also appears in SEL-88-002, Collected 
Software Engineering Papers: Volume VI . November 1988. 
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"SOFTWARE PROCESS AND PRODUCT SPECIFICATIONS: A BASIS 

FOR GENERATING CUSTOMIZED SE INFORMATION BASES," 

H. D. ROMBACH AND L. MARK, PROCEEDINGS OF THE 2 2ND 
ANNUAL HAWAII INTERNATIONAL CONFERENCE ON SYSTEM 
SCIENCES . JANUARY 1989, 10 PAGES 

This paper presents the software engineering oriented part 
of the “Meta Information Base for Software Engineering" 
project at the University of Maryland. The aim of this 
project is to generate customized software engineering 
information bases from formal specifications of software 
engineering processes and products. Systematic improvement 
of software processes and products, learning about software 
engineering approaches and reusing software engineering 
related experience, cannot be achieved without having a 
specification of the objects to be improved. This paper 
discusses general requirements for software process speci- 
fication languages, presents a first prototype software 
process specification language, demonstrates the application 
of this language and derives software engineering related 
requirements for a supporting information base. The actual 
efforts aimed at implementing these information base re- 
quirements are briefly mentioned in the conclusion. JAO 


This technical paper also appears in SEL-88-002, Collected 
Software Engineering Papers: Volume VI . November 1988. 
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5.24 CHARACTERIZING RESOURCE DATA: A MODEL FOR LOGICAL 

ASSOCIATION OF SOFTWARE DATA . D. R. JEFFERY AND 
V. R. BASILI , TR-1848, UNIVERSITY OF MARYLAND, 
TECHNICAL REPORT, MAY 1987, 35 PAGES 

This paper presents a conceptual model of software develop- 
ment resource data. A conceptual model, such as this, is a 
prerequisite to the development of integrated project sup- 
port environments that aim to assist in the processes of 
resource estimation, evaluation, and control. The model 
proposed is a four-dimensional view of resources that can be 
used for resource estimation, utilization, and review. A 
process model is presented showing the use pf the data 
model, and instances of the goal, question, metric paradigm 
are presented to show the applicability of the models to the 
measurement task. The model is validated by reference to 
published literature on resource data bases, and the impli- 
cations of the model in these data base environments is dis- 
cussed. SDB 


This paper also appears in SEL-87-009, Collected Software 
Engineering Papers: Volume V . November 1987. 
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TOWARDS A COMPREHENSIVE FRAMEWORK FOR REUSE: A REUSE- 

ENABLING SOFTWARE EVOLUTION ENVIRONMENT . V. BASILI AND 
H. ROMBACH, TR-2158, UNIVERSITY OF MARYLAND, TECHNICAL 
REPORT, DECEMBER 1988, 23 PAGES 

This paper motivates and outlines the scope of a comprehen- 
sive framework for understanding, planning, evaluating, and 
motivating reuse practices and the necessary research activ- 
ities. As a first step toward such a framework, a reuse- 
enabling software evolution environment model is introduced, 
which provides a basis for the effective recording of ex- 
perience, the generalization and tailoring of experience, 
the formalization of experience, and the (re-)use of experi- 
ence . SDB 


This technical paper also appears in SEL-89-006, Collected 
Software Engineering Papers: Volume VII . November 1989. 
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5.26 MAINTENANCE = REUSE-ORIENTED SOFTWARE DEVELOPMENT . 

V. BASILI , TR-2244, UNIVERSITY OF MARYLAND, TECHNICAL 

REPORT, MAY 1989, 12 PAGES 

This paper views maintenance as a reuse process and, in this 
context, discusses a set of models that can be used to sup- 
port the maintenance process. It presents a high-level re- 
use framework that characterizes the object of reuse, the 
process for adapting that object for its target application, 
and the reuse object within its target application. Based 
on this framework, a qualitative comparison is presented of 
the three maintenance process models, with regard to their 
strengths and weaknesses, and the circumstances in which 
they are appropriate. Providing a more systematic, quanti- 
tative approach for evaluating the appropriateness of the 
particular maintenance model, a measurement scheme, based on 
the reuse framework, is presented in the form of an orga- 
nized set of questions that need to be answered. A set of 
reuse enablers are discussed to support the reuse perspec- 
tive. SDB 


This technical paper also appears in SEL-89-006, Collected 
Software Engineering Papers: Volume VII . November 1989. 


8978 


5-28 


5.27 SOFTWARE DEVELOPMENT: A PARADIGM FOR THE FUTURE . 

V. BASILI, TR-2263 , UNIVERSITY OF MARYLAND, TECHNICAL 

REPORT, JUNE 1989, 29 PAGES 

This paper offers a new paradigm for software development 
that treats software development as an experimental activity 
The paradigm provides built-in mechanisms for learning how 
to better develop software and for reusing experience in the 
forms of knowledge, processes, and products. Models and 
measures are used to aid in characterization, evaluation, 
and motivation. An organization scheme is proposed for se- 
parating the project-specific focus from the organization's 
learning and reuse focuses of software development. The 
paper discusses the implications of this approach for cor- 
porations, research, and education and presents some re- 
search activities currently underway at the University of 
Maryland that support this approach. SDB 


This technical paper also appears in SEL-89-006, Collected 
Software Engineering Papers: Volume VII . November 1989. 
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5.28 TOWARDS A COMPREHENSIVE FRAMEWORK FOR REUSE: 
MODEL-BASED REUSE CHARACTERIZATION SCHEMES. 


V. BASILI AND H. ROMBACH , TR-2446, UNIVERSITY 
OF MARYLAND, TECHNICAL REPORT, APRIL 1990, 33 PAGES 

This technical report discusses the key role of reuse in 
enabling the software industry to achieve the dramatic im- 
provements in productivity and quality required to meet 
growing demands. Starting from assumptions about both soft- 
ware reuse and the software development process, it develops 
the characteristics required of successful reuse models. 

The report examines state-of-the-art reuse characterization 
schemes to evaluate how well they meet the required charac- 
teristics. It then defines a model-based reuse character- 
ization scheme, demonstrates the applicability of the scheme 
by applying it to three reuse scenarios, and finally pre- 
sents a reuse-oriented software environment model. JAO 


This technical paper also appears in SEL-90-005, Collected 
Software Engineering Papers: Volume VIII . November 1990. 
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5.29 “VIEWING MAINTENANCE AS REUSE-ORIENTED SOFTWARE 
DEVELOPMENT, “ V. BASILI, IEEE SOFTWARE . JANUARY 
1990, 7 PAGES 

This technical paper examines software maintenance as 
reuse-oriented development. It presents several maintenance 
models, culminating with a full-reuse model. The paper in- 
troduces a framework for reuse and compares the maintenance 
models and their application in the reuse framework. It con 
eludes with a discussion of support mechanisms that enable 
reuse, including the goal/question/metric improvement para- 
digm. JAO 


This technical paper also appears in SEL-90-005, Collected 
Software Engineering Papers; Volume VIII . November 1990. 
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5.30 "SOFTWARE REUSE: A KEY TO THE MAINTENANCE PROBLEM," 

H. D. ROMBACH, BUTTERWORTH JOURNAL OF INFORMATION AND 
SOFTWARE TECHNOLOGY . JANUARY/FEBRUARY 1991, 7 PAGES 

This paper describes software maintenance, points out that 
maintenance is inherently reuse oriented, and identifies 
some crucial maintenance problems. It then discusses 
similar problems in the area of software reuse, presents a 
comprehensive framework that has been proposed to address 
the reuse problems, and suggests how software maintenance 
may benefit by adopting a reuse-oriented framework. The 
paper concludes with an overview of reuse-related research 
at the University of Maryland. JAO 


This technical paper also appears in SEL-91-005, Collected 
Software Engineering Papers: Volume IX . November 1991. 
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5.31 SUPPORT FOR COMPREHENSIVE REUSE . V. R. BASILI AND 

H. D. ROMBACH , TR-2606 , UNIVERSITY OF MARYLAND, TECH- 
NICAL REPORT, FEBRUARY 1991, 40 PAGES 

This paper presents a comprehensive framework for reuse 
consisting of a reuse model, a model-based characterization 
scheme, and the Tailoring A Measurement Environment (TAME) 
model describing the integration of reuse into the software 
development process. Three hypothetical reuse scenarios — 
generic Ada packages (product reuse), design inspections 
(process reuse), and cost models (knowledge reuse) — are used 
to illustrate the approach and applicability of the proposed 
model and characterization scheme. 

A number of assumptions regarding software development in 
general and reuse in particular have resulted from more than 
15 years of analyzing software processes and products. 

These assumptions have led to the identification of four 
requirements essential for any useful reuse model and 
related characterization scheme. The paper identifies 
several existing reuse models and characterization schemes 
and illustrates that they only partially satisfy the 
requirements. A new reuse model that satisfies all the 
requirements is introduced and refined to derive a scheme 
for characterizing reuse candidates, reuse needs, and the 
reuse process. According to the proposed model, effective 
reuse requires an environment that supports continuous 
improvement. The remainder of the paper discusses the 
reuse-oriented model proposed by the TAME, compares the TAME 
model with the proposed reuse model, describes mechanisms 
for effective reuse in the context of the TAME model, and 
discusses aspects of the TAME research prototypes. SDB 


This technical paper also appears in SEL-91-005, Collected 
Software Engineering Papers: Volume IX , November 1991. 
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5.32 A REFERENCE ARCHITECTURE FOR THE COMPONENT FACTORY . 

V. R. BASILI AND G. CALDIERA, TR-2607, UNIVERSITY OF 

MARYLAND, TECHNICAL REPORT, MARCH 1991, 29 PAGES 

This paper explores an approach for increasing the quality 
and productivity of software development through reuse that 
expands on the traditional reuse of code to the reuse of 
software objects, their ' relationships , and associated 
experience. For reuse to be easy and effective, an 
organizational structure is needed that is flexible and 
allows for continuous improvement. This paper proposes an 
organizational framework that separates project-specific 
activities into a project organization and reuse packaging 
activities into an experience factory; these two groups 
interact, but their methods and tools are independent. The 
experience factory is further divided into subgroups: the 
domain factory and the component factory. The component 
factory provides reusable software components (RSCs) to 
projects upon demand and creates and maintains a repository 
of those components for future use. An RSC is defined to be 
not just code, but code packaged with everything necessary 
for its reuse and maintenance in an application system. The 
organizational framework is represented by a reference 
architecture using different levels of abstraction to obtain 
a flexible and evolutionary organizational design. This 
paper outlines a reference architecture and discusses the 
instantiation process, a methodology for deriving a 
particular environment from the general one. Some 
theoretical examples and a real case study are presented to 
illustrate the concept of reference architecture and the 
instantiation methodology. SDB 


This technical paper also appears in SEL-91-005, Collected 
Software Engineering Papers: Volume IX . November 1991. 
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5.33 A PATTERN RECOGNITION APPROACH FOR SOFTWARE ENGINEERING 
DATA ANALYSIS . L. C. BRIAND, V. R. BASILI, AND 
W. M. THOMAS, TR-2672/ UNIVERSITY OF MARYLAND, TECHNI- 
CAL REPORT, MAY 1991, 37 PAGES 

Managing a large-scale software development project requires 
the use of quantitative models to provide insight from the 
historical data of similar projects. This paper identifies 
how such models can be used to predict, understand, 
evaluate, and control the software development process, and 
defines the requirements for an effective data analysis 
procedure. It discusses the use of statistical analysis in 
the software engineering field and why the classical 
approaches do not meet most of the requirements for an 
effective data analysis procedure. The paper presents a new 
approach. Optimized Set Reduction (OSR) , which is based on 
pattern recognition techniques tailored to the software 
engineering field, and which offers advantages that overcome 
many of the problems associated with classical statistical 
analyses. It provides experimental results to demonstrate 
the effectiveness of the OSR approach for cost estimation 
modeling, discusses issues related to data analysis 
disturbances, and shows how an OSR approach might deal with 
them. The paper also describes the Improvement Paradigm and 
shows how the OSR performs with the learning and model 
refinement issues in the Improvement Paradigm framework. 
Finally, this paper presents eight positive characteristics 
of OSR that allow prediction, risk assessment, and quality 
evaluation, and identifies future research directions for 
work on the OSR approach. SDB 


This technical paper also appears in SEL-91-005, Collected 
Software Engineering Papers: Volume IX . November 1991. 
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5.34 SOFTWARE ENGINEERING LABORATORY l SEL) CLEANROOM 

PROCESS MODEL . SEL-9 1-004, S. GREEN, NOVEMBER 1991, 

74 PAGES 

This document describes the Software Engineering Laboratory 
(SEL) cleanroom process model. The model is based on data 
and analysis from previous cleanroom efforts within the SEL 
and is tailored to serve as a guideline in applying the 
methodology to future software production efforts. It 
describes the phases that are part of the process model life 
cycle from the delivery of requirements to the start of 
acceptance testing. For each defined phase, a set of 
specific activities is discussed, and the appropriate data 
flow is described. This document also presents pertinent 
managerial issues, key similarities and differences between 
the SEL ' s cleanroom process model and the standard 
development approach used on SEL projects, and significant 
lessons learned from prior cleanroom projects. It is 
intended that the process model described in this document 
will be further tailored as additional SEL cleanroom 
projects are analyzed. NTIS 
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6.1 "DESIGNING A SOFTWARE MEASUREMENT EXPERIMENT," 

V. R. BAS IL I AND M. V. ZELKOWITZ, PROCEEDINGS OF THE 
SOFTWARE LIFE CYCLE MANAGEMENT WORKSHOP . SEPTEMBER 1977, 
13 PAGES 

This technical paper explains the research approach employed 
by the SEL to study the development of actual software de- 
velopment projects. The following types of experiments are 
performed by the SEL: 

• Screening 

• Semicontrolled 

• Controlled 

This paper discusses these experimental designs, potential 
confounding effects, and the statistical techniques used to 
evaluate results. The effects on software developers of 
both learning during the experiment and an awareness of the 
experimental process itself are examined in detail. Fully 
controlled experiments are especially difficult to implement 
in a production environment, but sufficient control is pos- 
sible to evaluate the effects of software development meth- 
odologies. JAO 
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6.2 "ANALYZING MEDIUM SCALE SOFTWARE DEVELOPMENT," 

V. R. BASILI AND M. V. ZELKOWITZ, PROCEEDINGS OF THE 
THIRD INTERNATIONAL CONFERENCE ON SOFTWARE ENGINEERING . 
NEW YORK: IEEE COMPUTER SOCIETY PRESS, 1978, 8 PAGES 

This technical paper surveys SEL research activities in 
software engineering. The collection and analysis of data 
from software development projects is necessary for the de- 
finitive evaluation of software engineering methodologies 
and techniques. This paper describes the structure of the 
SEL and some of the early projects that were monitored. It 
also discusses the application of this data to resource 
utilization models and reliability studies. The principal 
contribution of the SEL, as reported in the paper, is the 
establishment of a facility for collecting the detailed data 
necessary for these analyses. JAO 
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6.3 "MEASURING SOFTWARE DEVELOPMENT CHARACTERISTICS IN THE 
LOCAL ENVIRONMENT," V. R. BASIL I AND M. V. ZELKOWITZ, 
COMPUTERS AND STRUCTURES . AUGUST 1978, VOL. 10, 5 PAGES 

This technical paper discusses the role of data collection 
in forecasting and monitoring software development projects 
in a production environment. The specific procedures of the 
SEL are reviewed, and SEL data collection forms are de- 
scribed. The paper also gives some examples of analyses 
that can be performed to support managing, understanding, 
and characterizing software development. The sample anal- 
yses identify specific measures for collection. JAO 


This technical paper also appears in SEL-82-004, Collected 
Software Engineering Papers: Volume I . July 1982. 
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6.4 


"EVALUATING AUTOMATABLE MEASURES FOR SOFTWARE DEVELOP- 
MENT," V. R. BASILI AND R. REITER, PROCEEDINGS OF THE 
WORKSHOP ON QUANTITATIVE SOFTWARE MODELS FOR RELI- 
ABILITY. COMPLEXITY. AND COST . NEW YORK: IEEE COM- 

PUTER SOCIETY PRESS, 1979, 10 PAGES 

This technical paper describes an approach to developing and 
evaluating automatable software measures. The experience of 
the SEL has shown that software data collection is an expen- 
sive activity that can significantly affect the software 
development process. Costs and effects can be minimized and 
data quality can be improved by automating the collection of 
measures wherever possible. 

This paper presents a set of automatable measures that were 
implemented and evaluated in a controlled experiment. The 
measures include computer job steps, program changes, pro- 
gram size, and software complexity. The results of the ex- 
periment indicate that the automated collection of these 
measures can be implemented effectively in production envi- 
ronments. JAO 
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6.5 "PROGRAMMING MEASUREMENT AND ESTIMATION IN THE SOFTWARE 
ENGINEERING LABORATORY," V. R. BASILI AND K. FREBURGER, 
JOURNAL OF SYSTEMS AND SOFTWARE . FEBRUARY 1981, VOL. 2, 
NO. 1, 11 PAGES 

This technical paper presents an examination of a set of 
basic relationships among various software development meas- 
ures, including size, effort, project duration, staff size, 
and productivity. Correlations among these measures are 
computed. The data used comes from 15 flight dynamics soft- 
ware development projects studied by the SEL. Certain rela- 
tionships are derived in the form of equations, and these 
equations are compared with a set derived by Walston and 
Felix for IBM Federal Systems Division project data. Loga- 
rithmic transformations were performed on the data for some 
analyses. Although the equations do not have the same coef- 
ficients, they are seen to have similar exponents. In fact, 
the SEL-derived equations tend to be within one standard 
error of the estimates provided in the IBM equations. JAO 


This technical paper also appears in SEL-82-004, Collected 
Software Engineering Papers: Volume I . July 1982. 
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6.6 "EVALUATING AND COMPARING SOFTWARE METRICS IN THE SOFT- 
WARE ENGINEERING LABORATORY," V. R. BASILI AND 
T. PHILLIPS, PROCEEDINGS OF THE ACM SIGMETRICS 
SYMPOSIUM/WORKSHOP: QUALITY METRICS . MARCH 1981, 

19 PAGES 

This technical paper describes an effort to identify the 
best measures of software development effort and software 
complexity. Four software projects studied by the SEL pro- 
vide the data for the analysis. The data is screened to 
ensure its validity. Next, estimating equations are derived 
for effort and errors using the various measures studied in 
the analysis. Correlations are shown to increase as the 
reliability of the data increases due to screening. Thus, a 
procedure is demonstrated for removing noise from the data 
and making possible meaningful comparisons of software 
metrics. JAO 


This technical paper also appears in SEL-82-004, Collected 
Software Engineering Papers: Volume I . July 1982. 
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6.7 "EARLY ESTIMATION OF RESOURCE EXPENDITURES AND 

PROGRAM SIZE," D. N. CARD, COMPUTER SCIENCES CORPORA- 
TION, TECHNICAL MEMORANDUM, JUNE 1982, 24 PAGES 

This technical memorandum evaluates the suitability of sev- 
eral software measures as estimators of resource expendi- 
tures and program size early in the software life cycle. 

The estimating equation based on the most commonly employed 
measure, lines of source code, is explained and its limita- 
tions are identified. Several alternative measures are in- 
vestigated and found to give good results. The memorandum 
also includes computer-generated output of the least-squares 
regression analyses upon which the conclusions are based. 

SDB 


This technical paper also appears in SEL-83-003, Collected 
Software Engineering Papers: Volume II . November 1983. 
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6.8 EVALUATION OF MANAGEMENT MEASURES OF SOFTWARE DEVELOP- 
MENT . SEL-82-001 , G. PAGE, D. N. CARD, AND 
F. E. McGARRY, SEPTEMBER 1982, VOL. 1: 143 PAGES, 

VOL. 2: 379 PAGES 

This two-volume document reports the results of an evalua- 
tion of a large set of software development measures rele- 
vant to the GSFC environment. The purposes of the analysis 
were to characterize the current software development proc- 
ess in one environment by identifying important qualities 
and corresponding measures and to evaluate the effectiveness 
of specific tools and techniques in this environment. The 
measures studied were counts, ratios, and management- 
supplied ratings of various elements of the software devel- 
opment process. The measures are high level in that each 
describes some aspect of an entire software project (or a 
large part of it) rather than individual components of the 
project. 

Volume 1 explains a conceptual model of software develop- 
ment, the data classification scheme, and the analytic pro- 
cedures. Factor analysis, cluster analysis, and a test of 
normality were used. This volume summarizes the results of 
those analyses and recommends specific software measures for 
collection and monitoring. Volume 1 also reproduces in full 
the results of the computer analyses. 

Volume 2 presents a detailed description of the data ana- 
lyzed, including definitions of measures, lists of values, 
and summary statistics. Although the information contained 
in Volume 2 was essential to the development of the explana- 
tion and summary presented in Volume 1, it is not essential 
to the understanding of that explanation and summary. How- 
ever, Volume 2 is useful in its own right as a source of 
data and a reference for future research. This document was 
also issued as Computer Sciences Corporation document 
CSC/TM- 82/6063. SDB 
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"METRIC ANALYSIS AND DATA VALIDATION ACROSS FORTRAN 
PROJECTS," V. R. BASILI, R. W. SELBY, AND T. PHILLIPS, 
IEEE TRANSACTIONS ON SOFTWARE ENGINEERING . NOVEMBER 
1983 

This technical paper reports the results of an analysis of 
the relationship of Halstead measures, McCabe complexity 
measures, and other software measures to software develop- 
ment effort and errors. Effort is defined in terms of 
staff-hours, from the establishing of functional specifica- 
tions through acceptance testing. Errors are counted dis- 
cretely and weighed according to effort to correct. The 
data studied was collected by the SEL in a production en- 
vironment. Cross-checks of the data indicated a need for 
large-scale data validation. The strongest correlations 
were obtained when the modules of individual programmers 
were considered independently. However, neither Halstead’s 
effort measure, McCabe's cyclomatic complexity measure, nor 
lines of source code was convincingly more accurate as an 
estimator than the others. This document was also issued as 
University of Maryland Technical Report TR-1228. JAO 


This technical paper also appears in SEL-83-003, Collected 
Software Engineering Papers: Volume II . November 1983. 
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6.10 "MEASURING SOFTWARE TECHNOLOGY," W. W. AGRESTI , 

F. E. McGARRY, D. N. CARD, ET AL., PROGRAM TRANS- 
FORMATION AND PROGRAMMING ENVIRONMENTS . NEW YORK: 
SPRINGER-VERLAG, 1984, 6 PAGES 

This paper summarizes the results of several recent SEL re- 
search efforts. The areas of software engineering discussed 
are programmer productivity, cost models, and technology 
evaluations. This paper stresses the importance of estab- 
lishing an organizational memory to provide a reference for 
evaluating software engineering techniques. The SEL data 
collection program is outlined as an example of such a mech- 
anism. JAO 


This technical paper also appears in SEL-83-003, Collected 
Software Engineering Papers: Volume II , November 1983. 
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6.11 


"SOFTWARE ERRORS AND COMPLEXITY: AN EMPIRICAL INVESTI 

GATION, " V. R. BASILI AND B. T. PERRICONE, COMMUNICA- 
TIONS OF THE ACM . JANUARY 1984, 11 PAGES 

This paper reports the results of an analysis of error data 
obtained from a flight dynamics software project studied by 
the SEL. The distributions of errors by type and location 
are identified and discussed. Correlations among module 
size, complexity, and efror rate are then described and - 
evaluated. Modified and new modules are shown to have simi- 
lar error characteristics. An alternative error-classif ica- 
tion scheme is developed. Finally, an attempt is made to 
compare these results with those of other researchers in the 
field. SDB 


This technical paper also appears ,in SEL-83-003, Collected 
Software Engineering Papers: Volume II . November 1983, and 

was also issued as University of Maryland Technical Report, 
TR-1195 , August 1982. 
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6.12 MEASURES AND METRICS FOR SOFTWARE DEVELOPMENT . 

SEL-83-002, D. N. CARD, F. E. McGARRY , G. PAGE, 

ET AL., MARCH 1984, 80 PAGES 

This document reports the evaluations of and recommendations 
for the use of software development measures based on the 
practical and analytical experience of the SEL. It de- 
scribes the basic concepts of measurement and a system of 
classification for measures. The principal classes of meas- 
ures defined are explicit, analytic, and subjective. Some 
of the major software measurement schemes appearing in the 
literature are reviewed. The applications of specific meas- 
ures in a production environment are explained. These ap- 
plications include the following: 

• Prediction and Planning 

• Review and Assessment 

• Evaluation and Selection 

An appendix describes the use of software development his- 
tories to manage ongoing software development projects. This 
document was also issued as Computer Sciences Corporation 
document CSC/TM-83/6061 . NTIS 
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6.13 "CHARACTERISTICS OF FORTRAN MODULES," D. N. CARD, 

Q. L. JORDAN, AND V. E. CHURCH, COMPUTER SCIENCES 
CORPORATION, TECHNICAL MEMORANDUM, JUNE 1984, 62 PAGES 

This study analyzes the characteristics of a large sample of 
FORTRAN modules produced by professional programmers. The 
proper organization and content of a software module are 
basic concerns of software developers^ Although many strat- 
egies and standards for software development are in use, few 
are based on any empirical evidence. The data studied was 
collected by the SEL. It includes error reports, staff 
charges, and source code measures. This study attempts to 
determine whether or not structural and quality differences 
exist among different classes of software and how knowledge 
of structural characteristics can be used to maximize soft- 
ware quality. JAO 
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6.14 STRUCTURAL COVERAGE OF FUNCTIONAL TESTING . TR-1442, 

V. R. BASILI AND J. RAMSEY, UNIVERSITY OF MARYLAND, 
TECHNICAL REPORT, SEPTEMBER 1984 

This technical report describes a study directed at under- 
standing and improving the acceptance test process in the 
NASA/GSFC SEL environment. A large, commercially developed 
FORTRAN program was modified to produce structural coverage 
metrics. The modified program was executed on a set of 
functionally generated acceptance tests and a large sample 
of operational usage cases. The resulting structural cov- 
erage metrics are combined with fault and error data to 
evaluate structural coverage in the SEL environment. 

It is shown that, in this environment, the functionally gen- 
erated tests seem to be a good approximation of operational 
use. The relative proportions of the exercised statement 
subclasses (executable, assignment, CALL, DO, IF, READ, 
WRITE) change as the structural coverage of the program in- 
creases. A method is proposed for determining whether two 
sets of input data exercise a program in a similar manner. 

Evidence is also provided implying that, in this environ- 
ment, faults revealed in a procedure are independent of the 
number of times the procedure is executed and that it may be 
reasonable to use procedure coverage in software models that 
use statement coverage. Finally, the evidence suggests that 
it may be possible to use structural coverage to aid the 
management of the acceptance test process. SDB 
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6.15 INVESTIGATION OF SPECIFICATION MEASURES FOR THE 
SOFTWARE ENGINEERING LABORATORY . SEL-84-003, 

W. W. AGRESTI , V. E. CHURCH, AND F. E. McGARRY, 
DECEMBER 1984 

This document presents an investigation of requirements 
specification measures for potential application in the 
SEL. Eighty-seven candidate measures are defined; sixteen 
are recommended for use. Most measures are derived from a 
new representation, the Composite Specification Model (CSM) , 
which is introduced in this document. CSM incorporates 
functional, entity/relationship, and state machine views of 
software requirements. The results of extracting the 
specification measures from the requirements of a real 
system are described. NTIS 
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6.16 "CRITERIA FOR SOFTWARE MODULARI ZATION , " D. N. CARD, 

G. PAGE, AND F. E. McGARRY, PROCEEDINGS OF THE EIGHTH 
INTERNATIONAL CONFERENCE ON SOFTWARE ENGINEERING . 

NEW YORK: IEEE COMPUTER SOCIETY PRESS, 1985, 6 PAGES 

This paper reports an attempt to determine the effectiveness 
of two widely used criteria for software modularization, 
strength and size, in reducing fault rate and development 
cost. The study was prompted by a central issue in prog- 
ramming practice that involves determining the appropriate 
size and information content of a software module. Data 
from 453 FORTRAN modules developed by professional program- 
mers were analyzed. The results indicated that module 
strength is a good criterion with respect to fault rate, 
whereas arbitrary module size limitations inhibit programmer 
productivity. This analysis is a first step toward defining 
empirically based standards for software modularization. JAO 


This technical paper also appears in SEL-85-003, Collected 
Software Engineering Papers: Volume III . November 1985. 
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6.17 


"CALCULATION AND USE OF AN ENVIRONMENT'S CHARACTER- 
ISTIC SOFTWARE METRIC SET," V. R. BASILI AND 
R. W. SELBY, JR., PROCEEDINGS OF THE EIGHTH 
INTERNATIONAL CONFERENCE ON SOFTWARE ENGINEERING . 

NEW YORK: IEEE COMPUTER SOCIETY PRESS, 1985, 

6 PAGES 

This paper presents an approach for customizing a character 
istic set of software metrics to an environment, since both 
cost/quality goals and production environments differ. The 
approach is applied in the SEL to 49 candidate process and 
product metrics of 652 modules from six projects (of 51,000 
to 112,000 lines). For this particular environment, the 
method yielded the characteristic metric set (source lines, 
fault correction effort per executable statement, design 
effort, code effort, number of I/O parameters, number of 
versions) . The uses examined for a characteristic metric 
set include forecasting the effort for development, modifi- 
cation, and fault correction of modules based on historical 
data. JAO 


This technical paper also appears in SEL-85-003, Collected 
Software Engineering Papers: Volume III . November 1985. 
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6.18 "EVALUATING SOFTWARE DEVELOPMENT BY ANALYSIS OF 

CHANGES: SOME DATA FROM THE SOFTWARE ENGINEERING 

LABORATORY , " D. M. WEISS AND V. R. BASILI, IEEE 
TRANSACTIONS ON SOFTWARE ENGINEERING . FEBRUARY 1985, 

12 PAGES 

This paper describes the application of an effective data 
collection methodology for evaluating software development 
methodologies to five different software development proj- 
ects. Results and data from three of the projects are pre- 
sented. Goals of the data collection included character- 
izing changes, errors, projects, and programmers; identi- 
fying effective error detection and correction techniques; 
and investigating ripple effects. 

The data collected consisted of changes (including error 
corrections) made to the software after code was written and 
baselined, but before testing began. Data collection and-., 
validation were concurrent with software development. 

Changes reported were verified by interviews with program- 
mers. Analysis of the data showed patterns that were used 
in satisfying the goals of the data collection. JAO 


This technical paper also appears in SEL-85-003, Collected 
Software Engineering Papers: Volume III , November 1985. 
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6.19 MEASURING SOFTWARE DESIGN . SEL-86-005, D. N. CARD, 

W. AGRESTI , V. CHURCH, ET AL., NOVEMBER 1986, 45 PAGES 

This paper describes extensive series of studies of software 
design measures conducted by the SEL. Included are the 
objectives and results of the studies, the method used to 
perform the studies, and the problems encountered. The 
document should be useful to researchers planning similar 
studies as well as to managers and designers concerned with 
applying quantitative design measures. NTIS 
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6.20 "A CONTROLLED EXPERIMENT ON THE IMPACT OF SOFTWARE 
STRUCTURE ON MAINTAINABILITY, " H. D. ROMBACH, IEEE 
TRANSACTIONS ON SOFTWARE ENGINEERING . MARCH 1987, 

11 PAGES 

This paper describes a study on the impact of software 
structure on maintainability aspects such as comprehensibil- 
ity, locality, modifiability, and reusability in a distrib- 
uted system environment. The study was part of a project at 
the University of Kaiserslautern, West Germany, to design 
and implement LADY, a LAnguage for Distributed sYstems. The 
study addressed the impact of software structure from two 
perspectives : 

• The language designer's perspective was to evaluate 
the general impact of the set of structural con- 
cepts chosen for LADY on the maintainability of 
software systems implemented in LADY. 

• The language user's perspective was to derive 
structural criteria (metrics), measurable from LADY 
systems, that allow the explanation or prediction 
of the software maintenance behavior. 

A controlled maintenance experiment was conducted involving 
12 medium-sized distributed software systems; 6 of these 
systems were implemented in LADY, the other 6 systems in an 
extended version of sequential Pascal. The benefits of the 
structural LADY concepts were judged based on a comparison 
of the average maintenance behavior of the LADY systems and 
the Pascal systems; the maintenance metrics were derived by 
analyzing the interdependence between structure and mainte- 
nance behavior of each individual LADY system. JAO 


This technical paper also appears in SEL-87-009, Collected 
Software Engineering Papers: Volume V . November 1987. 
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6.21 TAME: INTEGRATING MEASUREMENT INTO SOFTWARE ENVIRON- 

MENTS . V. R. BASIL I AND H. D. ROMBACH , TR-1764, 
UNIVERSITY OF MARYLAND, TECHNICAL REPORT, JUNE 1987, 

33 PAGES 

This paper describes the TAME project. Based upon a dozen 
years of analyzing software engineering processes and prod- 
ucts, we propose a set of software engineering process and 
measurement principles. These principles lead to the view 
that an integrated Software Engineering Environment (ISEE) 
should support multiple process models across the full soft- 
ware life cycle, the technical and management aspects of 
software engineering, and the planning, construction, and 
feedback and learning activities. These activities need to 
be tailored to the specific project under development, and 
they must be tractable for management control. The tailor- 
ability and tractability attributes require the support of a 
measurement process. The measurement process needs to be 
top-down, based upon operationally defined goals. 

The TAME project uses the goal/question/metric paradigm to 
support this type of measurement paradigm. It provides for 
the establishment of project-specific goals and corporate 
goals for planning software provides for the tracing of 
these goals throughout the life cycle via feedback and post 
mortem analysis, and offers a mechanism for long-range im- 
provement of all aspects of software development. 

The TAME system automates as much of this process as pos- 
sible by supporting goal development into measurement via 
models and templates, providing evaluation and analysis of 
the development and maintenance processes, and creating and 
using data bases of historical data and knowledge bases that 
incorporate experience from prior projects. SDB 


This technical paper also appears in SEL-87-009, Collected 
Software Engineering Papers: Volume V . November 1987. 
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6.22 "RESOURCE UTILIZATION DURING SOFTWARE DEVELOPMENT," 

M. V. ZELKOWITZ, THE JOURNAL OF SYSTEMS AND SOFTWARE . 

1988, 6 PAGES 

This paper discusses resource utilization over the life 
cycle of software development and discusses the role that 
the current “waterfall" model plays in the actual software 
life cycle. Software production in the NASA environment was 
analyzed to measure these differences. The results indicate 
that the waterfall model is not very realistic in practice, 
and that as technology introduces further perturbations to 
this model with concepts like executable specifications, 
rapid prototyping, and wide-spectrum languages, we need to 
modify our model of this process. JAO 


This technical paper also appears in SEL-88-002, Collected 
Software Engineering Papers: Volume VI . November 1988. 
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6.23 “VALIDATING THE TAME RESOURCE DATA MODEL," 

D. R. JEFFERY AND V. R. BASILI, PROCEEDINGS OF THE 
TENTH INTERNATIONAL CONFERENCE ON SOFTWARE ENGINEERING 
APRIL 1988, 15 PAGES 

This paper presents a conceptual model of software develop- 
ment resource data and validates the model by reference to 
the published literature on necessary resource data for 
development. _ support environments. The conceptual model 
presented here was developed using a top-down strategy. A 
resource data model is a prerequisite to the development of 
integrated project support environments that aim to assist 
in the processes of resource estimation, evaluation, and 
control. The model proposed is a four-dimensional view of 
resources that can be used for resource estimation, utiliza- 
tion, and review. This model is validated by reference to 
three publications on resource databases, and the implica- 
tions of the model arising out of these comparisons is dis- 
cussed. JAO 


This technical paper also appears in SEL-88-002, Collected 
Software Engineering Papers: Volume VI . November 1988. 
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"THE TAME PROJECT: TOWARDS IMPROVEMENT-ORIENTED SOFT- 

WARE ENVIRONMENTS," V. R. BASILI AND H. D. ROMBACH, 
IEEE TRANSACTIONS ON SOFTWARE ENGINEERING . JUNE 1988, 
16 PAGES 

This paper presents and discusses the improvement-oriented 
software engineering process model underlying the TAME 
(Tailoring A Measurement Environment) project, its automated 
support by the TAME system, and the first TAME system proto- 
type. The TAME project at the University of Maryland 
involves the development of an improvement-oriented software 
engineering process model that uses the goal/question/metric 
paradigm to integrate the constructive and analytic aspects 
of software development. The model provides a mechanism for 
formalizing the characterization and planning tasks, con- 
trolling and improving projects based on quantitative analy- 
sis, learning in a deeper and more systematic way about the 
software process and product, and feeding the appropriate 
experience back into the current and future projects. 

The TAME system is an instantiation of the TAME software 
engineering process model as an ISEE (Integrated Software 
Engineering Environment). The first in a series of TAME 
system prototypes has been developed. An assessment of 
experience with this first limited prototype is presented 
including a reassessment of its initial architecture. The 
long-term goal of this building effort is to develop a 
better understanding of appropriate ISEE architectures that 
optimally support the improvement-oriented TAME software 
engineering process model. JAO 


This technical paper also appears in SEL-88-002, Collected 
Software Engineering Papers: Volume VI . November 1988. 


8978 


6-25 


6.25 "MEASURING SOFTWARE DESIGN COMPLEXITY," D. N. CARD AND 

W. W. AGRESTI, THE JOURNAL OF SYSTEMS AND SOFTWARE . 

JUNE 1988, 13 PAGES 

This paper explains a new approach to measuring software 
design complexity that considers the structure of the over- 
all system as well as the complexity incorporated in individ- 
ual components. Architectural design complexity derives from 
two sources: structural (or intermodule) complexity and 

local (or intramodule) complexity. These complexity attrib- 
utes can be defined in terms of functions of the number of 
input/output variables and fanout of the modules comprising 
the design. A complexity indicator based on these measures 
showed good agreement with a subjective assessment of design 
quality but even better agreement with an objective measure 
of software error rate. Although based on a study of only 
eight medium-scale scientific projects, the data strongly 
support the value of the proposed complexity measure in this 
context. Furthermore, graphic representations of the soft- 
ware designs demonstrate structural differences corresponding 
to the results of the numerical complexity analysis. The 
proposed complexity indicator seems likely to be a useful 
tool for evaluating design quality before committing the 
design to code. JAO 


This technical paper also appears in SEL-88-002, Collected 
Software Engineering Papers: Volume VI . November 1988. 
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6.26 EVALUATING SOFTWARE DEVELOPMENT BY ANALYSIS OF CHANGE 
DATA . SEL-81-011, D. M. WEISS, NOVEMBER 1981, 

272 PAGES 

This document reports the results of an analysis of change 
data from five different software development projects in 
two different environments. A common data collection meth- 
odology was applied at both GSFC and the Naval Research Lab- 
oratory (NRL) . This document describes the data collection 
methodology employed, software projects studied, and the 
effects of changes on software development. 

The results of this study indicate that the data collection 
methodology is effective and easily extendable to new soft- 
ware development environments. Although the GSFC and NRL 
environments differed somewhat in their objectives and ap- 
proach to software development, the software produced by 
both groups was similar with respect to changes and errors. 
The results presented in this document include: (1) dis- 

tributions of causes of change, sources of errors, and dif- 
ficulty of finding errors and (2) tabulations of changes 
according to number of components changed, changes according 
to subsystem, difficulty of change (error) according to 
source of change (error) , and source of error according to 
programmer. SDB 
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6.27 EVALUATING SOFTWARE DEVELOPMENT BY ANALYSIS OF 

CHANGES: THE DATA FROM THE SOFTWARE ENGINEERING 

LABORATORY . SEL-82-008, V. R. BASILI AND D. M. WEISS, 
DECEMBER 1982, 77 PAGES 

This document reports the results of a study for evaluating 
software development by analyzing changes to the software. 
The specific goals of the study were to 

• Characterize changes and errors 

• Characterize projects and programmers 

• Identify effective error detection and correction 
techniques 

• Investigate ripple effects in the software caused 
by changes 

The data collected for the report consisted of changes (in- 
cluding error corrections) made to the software after code 
was written and baselined, but before testing began. Data 
collection and validation were concurrent with software de- 
velopment. Changes reported were verified by interviews 
with the originating programmers. Analysis of the data used 
in the study showed patterns that were used in satisfying 
the goals of the data collection. (Also see Section 5.3.) 

A version of this document was also issued as University of 
Maryland Technical Report TR-1236. NTIS 
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6.28 "A SUMMARY OF SOFTWARE MEASUREMENT EXPERIENCES IN THE 
SOFTWARE ENGINEERING LABORATORY," J. D. VALETT AND 
F. E. McGARRY, PROCEEDINGS OF THE 2 1ST ANNUAL HAWAII 
INTERNATIONAL CONFERENCE ON SYSTEM SCIENCES . 

JANUARY 1988, 9 PAGES 

This paper covers aspects of data collection and software 
measurement as they have been applied by one particular 
organization. The measurement results include the experi- 
ences and lessons learned through numerous experiments con- 
ducted by the SEL on nearly 60 flight dynamics software 
projects. These experiments have attempted to determine the 
effect of various software development technologies on over- 
all software project quality and on specific measures such 
as productivity, reliability, and maintainability. JAO 


This technical paper also appears in SEL-87-009, Collected 
Software Engineering Papers: Volume V . November 1987. 
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6.29 ESTABLISHING A MEASUREMENT BASED MAINTENANCE IMPROVE- 
MENT PROGRAM: LESSONS LEARNED IN THE SEL . H. ROMBACH 

AND B. ULERY, TR-2252 , UNIVERSITY OF MARYLAND, TECH- 
NICAL REPORT, MAY 1989, 27 PAGES 

This paper discusses the use of a goal-oriented approach to 
measurement to establish a maintenance improvement program 
within the SEL. Differences are found to exist between the 
initial phase of the program and its routine application. 
The approach is demonstrated through concrete examples, and 
lessons learned in the establishment of a measurement-based, 
maintenance improvement program are summarized. SDB 


This technical paper also appears in SEL-89-006, Collected 
Software Engineering Papers: Volume VII . November 1989. 
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6.30 INTEGRATING AUTOMATED SUPPORT FOR A SOFTWARE MANAGE- 
MENT CYCLE INTO THE TAME SYSTEM . T. SUNAZUKA AND 
V. BASILI , TR-2289 , UNIVERSITY OF MARYLAND, TECHNICAL 
REPORT, JULY 1989, 18 PAGES 

This paper discusses a metric setting procedure based on the 
Goal/Question/Metric (GQM) paradigm, a management system 
called the Software Management Cycle (SMC), and its applica- 
tion to a case study based on NASA/SEL data. The Tailoring 
A Measurement Environment (TAME) methodology, developed at 
the University of Maryland, is based on. the improvement 
paradigm and the GQM paradigm. The Software Quality Mea- 
surement and Assurance Technology (SQMAT), developed at NEC 
Corporation, is a software quality metric system and meth- 
odology applied to the development processes. TAME and SQMAT 
methodologies are integrated to realize goal-oriented mea- 
surement, process control, and visual management. The SMC 
is a substantiation of these concepts. The paper also de- 
scribes a method for evaluating the SMC process. The ex- 
pected effects of SMC are quality improvement, managerial 
cost reduction, accumulation and reuse of experience, and a 
highly visual management reporting system. SDB 


This technical paper also appears in SEL-89-006, Collected 
Software Engineering Papers: Volume VII . November 1989. 


8978 


6-31 


6.31 "DESIGN MEASUREMENT: SOME LESSONS LEARNED, “ 

H. ROMBACH , IEEE SOFTWARE . MARCH 1990, 9 PAGES 

This technical paper presents lessons learned about measure- 
ment in general and design measurement in particular. The 
lessons tend to fall into three categories: 

• How measurement must be applied in individual ex- 
periments or case studies 

• How measurement can help continuously improve an 
organization’s state of the practice 

• Why measurement requires automated support 

The lessons are illustrated with examples from the Distos/ 
Incas experiment, the SEL and the goal/question/metric para- 
digm, and the Tailoring a Measurement Environment (TAME) 

project. 

The paper discusses various design measurements and then 
proposes a comprehensive design measurement framework. JAO 


This technical paper also appears in SEL-90-005, Collected 
Software Engineering Papers: Volume VIII . November 1990. 
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6.32 SOFTWARE ENGINEERING LABORATORY ( SEL) RELATIONSHIPS . 

MODELS. AND MANAGEMENT RULES . S EL-9 1-001, W. DECKER, 

R. HENDRICK, AND J. VALETT, FEBRUARY 1991, 93 PAGES 

This document captures over 50 individual SEL research 
results, extracted from a review of published SEL 
documentation, that can be applied directly to managing 
software development projects. Four basic categories of 
results are defined and discussed: environment profiles, 

relationships, models, and management rules. In each 
category, research results are presented on a single page 
that summarizes the individual result, lists potential uses 
of the result by managers, and references the original SEL 
documentation where the result was found. The document 
serves as a concise reference summary of applicable research 
for SEL managers. NTIS 
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6.33 "PARADIGMS FOR EXPERIMENTATION AND EMPIRICAL STUDIES 

IN SOFTWARE ENGINEERING," V. R. BASILI AND R. W. SELBY 
RELIABILITY ENGINEERING AND SYSTEM SAFETY . 

JANUARY 1991, 21 PAGES 

Measurement techniques and empirical methods must be adopted 
by software researchers and practitioners to meet the high 
quality and productivity standards demanded by the complex 
systems of the future. This paper outlines the following 
four paradigms for experimentation and empirical study and 
describes their interrelationships: 

• Improvement Paradigm — A long-term, 
quality-oriented, organizational meta-life-cycle model that 
includes characterizing the environment, planning, 
execution, analysis, learning, and feedback. Use of this 
model by the Tailoring A Measurement Environment (TAME) 
project is discussed. 

• Goal/Question/Metric (GQM) Paradigm — A mechanism 
for defining and evaluating a set of operational goals using 
measurement on a specific project. 

• Experimentation Framework Paradigm — A refinement of 
the GQM paradigm that consists of four categories 
corresponding to phases of the experimentation process: 
definition, planning, operation, and interpretation. 

• Classification Paradigm — A specific instantiation 
of the improvement paradigm for product assessment involving 
data management and calibration, classification tree 
generation, and analysis and feedback. JAO 


This technical paper also appears in SEL-91-005, Collected 
Software Engineering Papers: Volume IX . November 1991. 
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SECTION 7 - TECHNOLOGY EVALUATIONS 
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7.1 A DEMONSTRATION OF AXES FOR NAVPAK . SEL-77-004, 

M. HAMILTON AND S. ZELDIN, SEPTEMBER 1977, 84 PAGES 

This document describes the results of a demonstration proj 
ect employing a system specification language tool to 
respond to a flight dynamics software development require- 
ment. The tool, AXES, was developed by Higher Order 
Software, Inc. Specific areas were selected for the demon- 
stration from the NAVPAK software requirements. The com- 
plexity of these requirements and the effort required to 
obtain a full understanding of them prevented the develop- 
ment of a complete design within the demonstration period. 
This document describes the following aspects of the demon- 
stration project: 

• Principles of HOS/AXES 

® Description of NAVPAK 

• HOS/AXES representation of NAVPAK specifications 

The appendixes contain some unrelated examples and other 
background material. This document was also issued as 
Higher Order Software, Inc., Technical Report 9. SDB 
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7.2 GSFC NAVPAK DESIGN SPECIFICATION LANGUAGES STUDY . 

SEL-77-005, P. A. SCHEFFER AND C. E. VELEZ, 

OCTOBER 1977, 37 PAGES 

This document reports the results of an analysis of the ap- 
plicability of software specification and design languages 
to the development of flight dynamics software at GSFC. Two 
language systems were studied: HOS/AXES and PSL/PSA. The 

approach was to develop software designs, using each of 
these language systems, from the reguirements of an already- 
implemented system. The results of the two methodologies 
were then compared with each other and with the actual de- 
sign implemented. This document was also issued as a Martin 
Marietta Corporation technical report. CAST 
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7.3 GSFC SOFTWARE ENGINEERING RESEARCH REQUIREMENTS 
ANALYSIS STUDY . SEL-78-006, P. A. SCHEFFER AND 
C. E. VELEZ , NOVEMBER 1978, 26 PAGES 

This document reports the results of a study of the applica- 
bility of requirements languages to flight dynamics software 
development at GSFC. The specific objectives of the study, 
which are explained in this document, were to 

• Determine the impact of requirements language use 
on software design 

• Demonstrate the application of a requirements lan- 
guage on a flight dynamics development problem 

• Evaluate the utility of the Multi-Level Expression 
Design Language - Requirement Level (MEDL-R) in the 
GSFC environment 

o Determine the desirable characteristics of a re- 
quirements language tool for use in the GSFC en- 
vironment 

This document was also issued as a Martin Marietta Corpora- 
tion technical memorandum. CAST 
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7.4 EVALUATION OF THE CAINE. FARBER. AND GORDON PROGRAM 
DESIGN LANGUAGE (PPL) IN THE GODDARD SPACE FLIGHT 
CENTER ( GSFC) CODE 580 SOFTWARE DESIGN ENVIRONMENT . 
SEL-79-004 , C. E. GOOREVICH, A. L. GREEN, AND 
W. J. DECKER, SEPTEMBER 1979, 46 PAGES 

This document reports the results of a study of the useful- 
ness of program design languages (PDLs) for flight dynamics 
software development at GSFC. The following PDLs were ex- 
amined and compared: 

• Telemetry Computation Branch PDL 

• Linger and Mills PDL 

• Caine, Farber, and Gordon PDL 

The last PDL was selected for intensive study. Its advan- 
tages and disadvantages in the flight dynamics environment 
were evaluated. Appendixes include examples of the use of 
the Caine, Farber, and Gordon PDL and the processor output. 
This document was also issued as Computer Sciences Corpora- 
tion document CSC/TM-79/6263 . CAS I 
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7.5 MULTI-LEVEL EXPRESSION DESIGN LANGUAGE - REQUIREMENT 
LEVEL (MEDL-R) SYSTEM EVALUATION . SEL-80-002, 

W. J. DECKER AND C. E. GOOREVICH, MAY 1980, 91 PAGES 

This document presents the results of an evaluation of the 
suitability of the Multi-Level Expression Design Language - 
Requirement Level (MEDL-R) for use in flight dynamics soft- 
ware development at GSFC. The evaluation team studied the 
MEDL-R concept of requirements languages, the functions per- 
formed by MEDL-R, and the MEDL-R language syntax. The docu- 
ment contains recommendations for changes to the MEDL-R 
system that would make it more useful in the flight dynamics 
environment. This document was also issued as Computer 
Sciences Corporation document CSC/TM-80/6093 . CAS I 
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7.6 MULTI -MISSION MODULAR SPACECRAFT GROUND SUPPORT SOFTWARE 

SYSTEM (MMS/GSSS) STATE-OF-THE-ART COMPUTER SYSTEMS/ 

COMPATIBILITY STUDY . SEL-80-003, T. WELDEN, 

m. McClellan, and p. liebertz, may 1980, 66 pages 

This document describes the results of an evaluation of the 
compatibility between the ModComp IV/35 and the VAX-11/780 
computers with respect to a specific software system, the 
Multi-Mission Modular Spacecraft Ground Support Software 
System (MMS/GSSS) . The degree of compatibility was measured 
by comparing the results of benchmark tests run on both sys- 
tems. The tests examined input/output services, FORTRAN 
language implementation, and execution timing. The compati- 
bility of peripheral devices and system command languages 
was considered in lesser detail. Significant incompatibil- 
ities were found in all areas examined. This document was 
also issued as Computer Sciences Corporation document 
CSC/TM-80/6154 . NTIS 
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7.7 "USE OF CLUSTER ANALYSIS TO EVALUATE SOFTWARE ENGINEER- 
ING METHODOLOGIES," E. CHEN AND M. V. ZELKOWITZ, 
PROCEEDINGS OF THE FIFTH INTERNATIONAL CONFERENCE ON 
SOFTWARE ENGINEERING . NEW YORK: IEEE COMPUTER SOCIETY 

PRESS, 1981, 7 PAGES 

This technical paper describes an attempt to identify the 
characteristic effects of various methodologies on software 
development. Data collected by the SEL from five software 
projects was studied. Several objective measures were de- 
rived from the data, and their relationships to methodology 
use were studied with cluster analysis techniques. The 
analysis showed that the measures reflected the effects of 
methodologies on software development. JAO 


This technical paper also appears in SEL-82-004, Collected 
Software Engineering Papers: Volume I . July 1982. 
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7.8 SOFTWARE ENGINEERING LABORATORY PROGRAMMER WORKBENCH 
PHASE 1 EVALUATION . SEL-8 1-009, W. J. DECKER AND 
F. E. McGARRY, MARCH 1981, 29 PAGES 

This document summarizes an initial effort to develop a pro- 
grammer workbench for flight dynamics software development 
activities. Phase 1 of the programmer workbench consists of 
the design of three components: the communications link, 

the command language processor, and the software tools pack- 
age. The document also contains a brief description and 
evaluation of the design of each component. Some recommen- 
dations for future work are made. This document was also 
issued as Computer Sciences Corporation document 
CSC/TM-81/609 1 . CAST 
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7.9 


“A SOFTWARE ENGINEERING VIEW OF THE FLIGHT DYNAMICS 
ANALYSIS SYSTEM (FDAS) : PARTS I AND II," D. N. CARD, 

W. W. AGRESTI, V. E. CHURCH, AND Q. L. JORDAN, COMPUTER 
SCIENCES CORPORATION, TECHNICAL MEMORANDUM, 

DECEMBER 1983 (PART I) AND MARCH 1984 (PART II), 

58 PAGES 

This report presents the results of an assessment, from the 
software engineering point of view, of the Flight Dynamics 
Analysis System (FDAS) at one step in the requirements- def- 
inition process — a prototype support environment. FDAS is 
intended to provide an integrated software development sup- 
port environment for research applications in the areas of 
orbit, attitude, and mission analysis, and it was conceived 
to assist users in the preparation, execution, and interpre- 
tation of software experiments. A prototype FDAS was con- 
structed to aid in clarifying the requirements for such a 
system and to test some concepts of language, software 
structure, and user interface designs. Part I of the report 
discusses the general approaches to FDAS adapted by the de- 
velopment team. Part II presents a detailed examination of 
some high-level FDAS design issues and summarizes some simi- 
lar systems from other environments. SDB 
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7.10 "A PRACTICAL EXPERIENCE WITH INDEPENDENT VERIFICATION 
AND VALIDATION, " G. PAGE, F. E. McGARRY, AND 
D. N. CARD, PROCEEDINGS OF THE EIGHTH INTERNATIONAL 
COMPUTER SOFTWARE AND APPLICATIONS CONFERENCE . 

NEW YORK: IEEE COMPUTER SOCIETY PRESS, 1984, 5 PAGES 

This paper describes an attempt to assess the benefits and 
limitations of the application of independent verification 
and validation (IV&V) in the flight dynamics area at NASA/ 
GSFC. The SEL applied the IV&V methodology to two medium- 
sized flight dynamics software development projects. Then, 
to measure the effectiveness of the IV&V approach, the SEL 
compared these two projects with two similar past projects, 
using measures like productivity, reliability, and maintain 
ability. Results indicated that the use of the IV&V method 
ology did not help the overall process nor improve the 
product in these cases. JAO 


This technical paper also appears in SEL-85-003, Collected 
Software Engineering Papers: Volume III . November 1985. 
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7.11 "ANALYZING THE TEST PROCESS USING STRUCTURAL COVERAGE , " 
J. RAMSEY and V. R. BASILI, PROCEEDINGS OF THE EIGHTH 
INTERNATIONAL CONFERENCE ON SOFTWARE ENGINEERING . 

NEW YORK: IEEE COMPUTER SOCIETY PRESS, 1985, 7 PAGES 

This paper reports the results of a study to understand and 
improve the acceptance test process in the SEL environment. 

An SEL program, the MAL language preprocessor (a subset of a 
satellite attitude maintenance system), has been modified to 
produce structural coverage metrics. It was modified to 
measure both procedure coverage and statement coverage. 
Coverage is also computed for several statement subclasses. 
The modified program was executed on a set of functionally 
generated acceptance tests and a large sample of operational 
usage cases. The resulting structural coverage metrics are 
combined with fault and error data to evaluate structural 
Coverage in the SEL environment. 

It is shown that, in this environment, the functionally gen- 
erated tests seem to be a good approximation of operational 
use. The relative proportions of the exercised statement 
subclasses change as the structural coverage of the program 
increases. A method is proposed for evaluating whether two 
sets of input data exercise a program in a similar manner. 
Evidence also shows that (1) faults revealed in a procedure 
are independent of the number of times the procedure is 
executed and (2) it may be reasonable to use procedure cov- 
erage in software models that use statement coverage. Fi- 
nally, the evidence suggests that it may be possible to use 
structural coverage to aid in managing the acceptance test 
process. JAO 


This technical paper also appears in SEL-85-003, Collected 
Software Engineering Papers: Volume III . November 1985, and 

in Structural Coverage of Functional Testing . University of 
Maryland Technical Report, TR-1442, September 1984. 


8978 


7-12 



7.12 "MEASURING THE IMPACT OF COMPUTER RESOURCE QUALITY ON 
THE SOFTWARE DEVELOPMENT PROCESS AND PRODUCT," 

F. E. McGARRY, J. VALETT, AND D. HALL, PROCEEDINGS 
OF THE HAWAII INTERNATIONAL CONFERENCE ON SYSTEM 
SCIENCES, JANUARY 1985, 9 PAGES 

This study examined the relationship between computer re- 
sources and the software development process and product as 
exemplified by NASA/GSFC data. Data have been extracted and 
examined from nearly 50 software development projects vary- 
ing in size from 3,000 to 130,000 lines of code. All have 
been related to the support of satellite flight dynamics 
ground-based computations. As a result of changing situa- 
tions and technology, the computer support environment has 
varied widely. Some projects enjoyed fast response time, 
excess memory, and state-of-the-art tools, whereas others 
endured slow computer response time, archaic tool support, 
and limited terminal access to the development machine. 

Based on the results of this study, a number of computer- 
resource-related implications are provided. JAO 


This technical paper also appears in SEL-85-003, Collected 
Software Engineering Papers: Volume III . November 1985. 
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7.13 COMPARISON OF SOFTWARE VERIFICATION TECHNIQUES . 

SEL-85-001, D. N. CARD, R. W. SELBY, F. E. McGARRY , 

ET AL., APRIL 1985, 90 PAGES 

This document describes a controlled experiment performed by 
the SEL to compare the effectiveness of code reading, func- 
tional testing, and structural testing as software verifica- 
tion techniques. It is one of a series of three experiments 
organized by R. W. Selby as part of his doctoral disserta- 
tion. The experiment results indicate that code reading 
provides the greatest error detection capability at the 
lowest cost, whereas structural testing is the least effec- 
tive technique. This document explains the experiment plan, 
describes the experiment results, and discusses related re- 
sults from other studies. It also considers the application 
of these results to the development of software in the flight 
dynamics environment. Appendixes summarize the experiment 
data and list the test programs. A separate Data Supplement 
contains original materials collected from the participants. 
NTIS 
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7.14 EVALUATIONS OF SOFTWARE TECHNOLOGIES: TESTING. CLEAN- 

ROOM. AND METRICS . SEL-85-004, R. W. SELBY, JR., 

MAY 1985, 183 PAGES 

This document describes a seven-step approach for quanti- 
tatively evaluating software technologies, coupling software 
methodology evaluation with software measurement. The 
approach is applied in depth in the following three areas: 

• Software Testing Strategies — A 74-subject study, 
including 32 professional programmers and 42 advanced uni- 
versity students, compared code reading, functional testing, 
and structural testing in a fractional factorial design. 

• Cleanroom Software Development — Fifteen 3-person 
teams separately built a 1200-line message system to compare 
Cleanroom software development (in which software is devel- 
oped completely off line) with a more traditional approach. 

• Characteristic Software Metric Sets — In the SEL 
production environment, a study of 65 candidate product' and 
process measures of 652 modules from 6 projects of 51,000 to 
112,000 lines yielded a characteristic set of software 
cost/quality metrics. SDB 


This technical paper was also issued as University of 
Maryland Technical Report, TR-1500, May 1985. 
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7.15 EVALUATION OF AN INDEPENDENT VERIFICATION AND VALIDA- 
TION (IV&V) METHODOLOGY FOR FLIGHT DYNAMICS . 

SEL— 81— 110 , G. PAGE, F. E. McGARRY, AND D. N. CARD, 
JUNE 1985, 53 PAGES 

This document describes an experiment in the application of 
an independent verification and validation (IV&V) methodol- 
ogy to the development of flight dynamics software at GSFC. 
IV&V is the systematic evaluation of computer software by an 
organization that is independent of the development organi- 
zation. IV&V is expected to provide earlier error detection 
and better quality control over the development process. 

This document describes the environment, staffing, and re- 
sults of the experiment. Costs and error rates are compared 
with those of similar projects developed without IV&V. An 
IV&V methodology is found to be appropriate for very large 
projects and for those with high reliability requirements. 
This document was also issued as Computer Sciences Corpora- 
tion document CSC/TM-85/6045 . SDB 


The previous version of this document was Performance and 
Evaluation of an Independent Software Verification and Inte- 
gration Process . SEL-81-010, G. Page and F. E. McGarry, May 
1981. 
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7.16 "FOUR APPLICATIONS OF A SOFTWARE DATA COLLECTION AND 
ANALYSIS METHODOLOGY," V. R. BASILI AND 
R. W. SELBY, JR., PROCEEDINGS OF THE NATO ADVANCED 
STUDY INSTITUTE . AUGUST 1985, 15 PAGES 

This paper presents a seven-step data collection and analy- 
sis methodology that couples software technology evaluation 
with software measurement. Four in-depth applications of 
the methodology are presented. The four studies represent 
each of the general categories of analyses on the software 
product and development process: blocked subject-project 

studies, replicated project studies, multiproject variation 
studies, and single project studies. The four applications 
are in the areas of software testing strategies, Cleanroom 
software development, characteristic software metric sets, 
and software error analysis, respectively. JAO 


This technical paper also appears in SEL-85-003, Collected 
Software Engineering Papers: Volume III . November 1985. 
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7.17 "QUANTITATIVE EVALUATION OF SOFTWARE METHODOLOGY," 

V. R. BASIL I , PROCEEDINGS OF THE FIRST PAN-PACIFIC 
COMPUTER CONFERENCE . SEPTEMBER 1985, 21 PAGES 

This paper presents a paradigm for evaluating software de- 
velopment methods and tools. The basic idea is to generate 
a set of goals that are refined into quantifiable questions. 
These questions specify the metrics to be collected on the 
software development and maintenance process and product. 

The metrics can be used to characterize, evaluate, predict, 
and motivate. They can be used in an active as well as pas- 
sive way by learning from analyzing the data and applying 
what is learned to improving the methods and tools employed 
in practice. Several examples were given representing each 
of the different approaches to evaluation. JAO 


This technical paper also appears in SEL-85-003, Collected 
Software Engineering Papers: Volume III . November 1985, and 

in University of Maryland Technical Report, TR-1519, July 
1985. 


8978 


7-18 


7.18 "A SOFTWARE TECHNOLOGY EVALUATION PROGRAM, " D. N. CARD, 

ANNA IS DO XVIII CONGRESSO NACIONAL DE INFORMATICA . 

OCTOBER 1985, 6 PAGES 

This paper describes an ongoing technology evaluation pro- 
gram conducted by the SEL that is intended to resolve cer- 
tain issues in the application of tools, practices, and 
techniques by software developers. A wealth of potentially 
beneficial software engineering tools, practices, and tech- 
niques has emerged in the past several years. Simulta- 
neously, realization has grown that all software engineering 
technologies are not equally effective for all software de- 
velopment problems and environments. The steps to tech- 
nology improvement include measurement, evaluation, and 
transference. The SEL collects measures on the production 
of FORTRAN software for spacecraft navigation systems. Re- 
cent SEL investigations demonstrated that the use of struc- 
tured programming and quality assurance improves software 
reliability. Also, intensive computer use appears to be 
associated with low productivity. However, the major factor 
in both productivity and reliability continues to be per- 
sonnel capability. Such technology evaluation programs pro- 
vide an empirical basis for defining software development 
standards and selecting tools. JAO 


This technical paper also appears in SEL-85-003, Col lected 
Software Engineering Papers: Volume III . November 1985. 

This material was also presented at the ACM Computer Science 
Conference, New Orleans, Louisiana, March 1985. 
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7.19 "AN EMPIRICAL STUDY OF SOFTWARE DESIGN PRACTICES, " 

D. N. CARD, V. E. CHURCH, AND W. W. AGRESTI, IEEE 
TRANSACTIONS ON SOFTWARE ENGINEERING . FEBRUARY 1986, 

8 PAGES 

This paper reports the results of an empirical study of 
software design practices in one specific environment. The 
practices examined affect module size, module strength, data 
coupling, descendant -span, unreferenced variables, and soft- 
ware reuse. Measures characteristic of these practices were 
extracted from 887 FORTRAN modules developed for five flight 
dynamics software projects monitored by the Software Engi- 
neering Laboratory. The relationship of these measures to 
cost and fault rate was analyzed using a contingency table 
procedure. The results show that some recommended design 
practices, despite their intuitive appeal, are ineffective 
in this environment, whereas others are very effective. JAO 


This technical paper also appears in SEL-86-004, Collected 
Software Engineering Papers: Volume IV . November 1986. 
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7.20 "AN APPROACH FOR ASSESSING SOFTWARE PROTOTYPES," 

V. E. CHURCH, D. N. CARD, W. W. AGRESTI , AND 
Q. L. JORDAN, ACM SOFTWARE ENGINEERING NOTES . 

JULY 1986, 12 PAGES 

This paper presents a procedure for evaluating a software 
prototype. The need to assess the prototype itself arises 
from the use of prototyping to demonstrate the feasibility 
of a design or development strategy. The assessment proce- 
dure can also be of use in deciding whether to evolve a pro- 
totype into a complete system. The procedure consists of 
identifying evaluation criteria, defining alternative design 
approaches, and ranking the alternatives according to the 
criteria. JAO 


This technical paper also appears in SEL-86-004, Collected 
Software Engineering Papers: Volume IV . November 1986. 
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7.21 "AN EVALUATION OF EXPERT SYSTEMS FOR SOFTWARE ENGINEER- 
ING MANAGEMENT," C. L. RAMSEY AND V. R. BASILI, 

IEEE TRANSACTIONS ON SOFTWARE ENGINEERING . JUNE 1989 
12 PAGES 

This report provides an evaluation of the use of expert sys- 
tems for software engineering management. Although the 
field of software engineering is relatively new, it can 
benefit from the use of expert systems. Four prototype ex- 
pert systems have been developed to aid in software engi- 
neering management. Given the values for certain metrics, 
these systems will provide interpretations that explain any 
abnormal patterns of these values during the development of 
a software project. The four expert systems, which solve 
the same problem, were built using two different approaches 
to knowledge acquisition, a bottom-up approach and a top- 
down approach, and two different expert system methods, 
rule-based deduction and frame-based abduction. A compari- 
son was performed to see which methods best suit the needs 
of this field. It was found that the bottom-up approach led 
to better results than did the top-down approach, and the 
rule-based deduction systems using simple rules provided 
more complete and correct solutions than did the frame-based 
abduction systems. JAO 


This paper also appears as a University of Maryland technical 
report in SEL-87-009, Collected Software Engineering Papers: 
Volume V . November 1987. 


8978 


7-22 


7.22 "THE EFFECTIVENESS OF SOFTWARE PROTOTYPING: A CASE 

STUDY, " M. V. ZELKOWITZ, PROCEEDINGS OF THE 26TH ANNUAL 
TECHNICAL SYMPOSIUM OF THE WASHINGTON. D.C. . CHAPTER OF 
THE ACM . JUNE 1987, 9 PAGES 

This paper discusses resource utilization over the life 
cycle of software development, and discusses the role that 
the current “waterfall model" plays in the actual software 
life cycle. The effects of prototyping are measured with 
respect to the life cycle model. Software production in the 
NASA environment was analyzed to measure these differences. 
The data collected from 13 different projects and 1 proto- 
type development were collected by the Software Engineering 
Laboratory and analyzed for similarities and differences. 

The results indicate that the waterfall model is not very 
realistic in practice and that a prototype development fol- 
lows a similar life cycle as a production system. JAO 


This technical paper also appears in SEL-88-002, Collected 
Software Engineering Papers: Volume VI . November 1988. 
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7.23 "EVALUATING SOFTWARE ENGINEERING TECHNOLOGIES," 

D. N. CARD, F. E. McGARRY, AND G. T. PAGE, IEEE TRANS- 
ACTIONS ON SOFTWARE ENGINEERING . JULY 1987, 6 PAGES 

The objectives of this study were to measure technology use 
in a production environment, develop a statistical model for 
evaluating the effectiveness of technologies, and evaluate 
the effects of some specific technologies on productivity 
and reliability. A carefully matched sample of 22 projects 
from the SEL data base was studied using an analysis-of- 
covariance procedure. Limited use of the technologies con- 
sidered in the analysis produced approximately a 30-percent 
increase in software reliability. These technologies did 
not demonstrate any direct effect on development produc- 
tion. JAO 


This technical paper also appears in SEL-87-009, Collected 
Software Engineering Papers: Volume V . November 1987. 
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7.24 


"QUANTITATIVE ASSESSMENT OF MAINTENANCE: AN INDUSTRIAL 

CASE STUDY," H. D. ROMBACH AND V. R. BASILI , PROCEED- 
INGS FROM THE CONFERENCE ON SOFTWARE MAINTENANCE . 
SEPTEMBER 1987, 11 PAGES 

This paper discusses a study aimed at the improvement of 
measurement and evaluation procedures used in an industrial 
maintenance environment. The following topics are discussed: 

• General measurement, evaluation, and improvement 
goals important to this environment 

• A set of metrics derived for quantifying those goals 

• Suggested changes to the current data collection 
procedures 

• Preliminary analysis results based on a limited set 
of already available data 

• Ideas for automating the proposed quantitative 
assessment approach 

This paper emphasizes the steps of introducing such a quan- 
titative maintenance approach into an industrial setting 
rather than the environment-specific analysis results. The 
analysis results are intended to demonstrate the practical 
applicability and feasibility of the proposed methodology 
for evaluating and improving maintenance aspects in an 
industrial environment. JAO 


This technical paper also appears in SEL-88-002, Collected 
Software Engineering Papers: Volume VI . November 1988. 
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7.25 "COMPARING THE EFFECTIVENESS OF SOFTWARE TESTING 
STRATEGIES , " V. R. BASILI AND R. W. SELBY, IEEE 
TRANSACTIONS ON SOFTWARE ENGINEERING . DECEMBER 1987, 

60 PAGES 

This study applies an experimentation methodology to compare 
three state-of-the-practice software testing techniques: 

(1) code reading by stepwise abstraction, (2) functional 
testing using equivalence partitioning and boundary value 
analysis, and (3) structural testing using 100-percent 
statement coverage criteria. The study compares the strat- 
egies in three aspects of software testing: fault detection 

effectiveness, fault detection cost, and classes of faults 
detected. Thirty-two professional programmers and 42 ad- 
vanced students applied the 3 techniques to 4 unit-sized 
programs in a fractional factorial experimental design. The 
major results of this study are the following. 

• With the professional programmers, code reading 
detected more software faults and had a higher fault detec- 
tion rate than functional or structural , testing did, while 
functional testing detected more faults than structural 
testing did, but functional and structural testing were not 
different in fault detection rate. 

• In one advanced student subject group, code reading 
and functional testing were not different in faults found 
but were both superior to structural testing, while in the 
other advanced student subject group there was no difference 
among the techniques. 

• With the advanced student subjects, the three 
techniques were not different in fault detection rate. 

• Number of faults observed, fault detection rate, 
and total effort in detection depended on the type of 
software tested. 
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• Functional testing detected more control faults 
than the other methods did. 

• When asked to estimate the percentage of faults 
detected, code readers gave the most accurate estimates while 
functional testers gave the least accurate estimates. SDB 


This technical paper also appears in SEL-87-009, Collected 
Software Engineering Papers: Volume V . November 1987, and 

as University of Maryland Technical Report TR-1501, May 1985. 
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7.26 “ ARROWSM I TH - P — A PROTOTYPE EXPERT SYSTEM FOR SOFTWARE 

ENGINEERING MANAGEMENT," V. R. BASILI AND 
C. L. RAMSEY, PROCEEDINGS OF THE IEEE/MITRE EXPERT 
SYSTEMS IN GOVERNMENT SYMPOSIUM . OCTOBER 1985, 13 PAGES 

This paper evaluates two prototype expert systems, collec- 
tively named ARROWSMITH-P . ARROWSMITH-P is intended to aid 
the manager of a software development project in an auto- 
mated manner. The systems work as follows. First, it is 
determined whether a software project is following normal 
development patterns by comparing measures such as pro- 
grammer hours per line of source code against historical, 
environment-specific baselines of such measures. The 
"manifestations" detected by this comparison, such as an 
abnormally high rate of programmer hours per line of source 
code, then serve as input to each expert system. Finally, 
each system attempts to determine the reasons, such as low 
productivity, for any abnormal software development pat- 
terns. These systems can be updated as the environment 
changes and as more is learned in the field of software 
engineering . 

The two systems, which solve the same problem, were built 
using different methods: rule-based deduction and frame- 

based abduction. A comparison was performed to determine 
which method better suits the needs of this field. It was 
found that both systems performed moderately well, but the 
rule-based deduction system using simple rules provided more 
complete solutions than did the frame-based abduction sys- 
tem. JAO 


This technical paper also appears in SEL-85-003, Collected 
Software Engineering Papers: Volume III . November 1985. 
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7.27 "EVOLUTION TOWARDS SPECIFICATIONS ENVIRONMENT: 
EXPERIENCE WITH SYNTAX EDITORS," M. ZELKOWITZ, 
INFORMATION AND SOFTWARE TECHNOLOGY . APRIL 1990, 

8 PAGES 

This technical paper discusses efforts to extend the inte- 
grated environment of Support, a syntax-directed editor, to 
the specification domain. Syntax-directed editors have been 
studied over the last 10 years and have not proven as effec- 
tive as originally thought. This paper analyzes the experi- 
ence of 3 years of student use of Support. In general, 
features that are useful to novice programmers tend to in- 
terfere with more experienced users. 

The paper describes two projects to extend Support into sys- 
tem specification. The first supports axiomatic specifica- 
tions, while the second supports functional specification. 
Further work is needed to test the applicability of this 
form of environment. JAO 



This technical paper also appears in SEL-90-005, Collected 
Software Engineering Papers: Volume VIII . November 1990. 
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7.28 THE CLEANROOM CASE STUDY IN THE SOFTWARE ENGINEERING 
LABORATORY: PROJECT DESCRIPTION AND EARLY ANALYSIS . 

SEL-90-002, S. GREEN ET AL., MARCH 1990, 65 PAGES 

This case study analyzes the application of the Cleanroom 
software development methodology to the development of pro- 
duction software at NASA/GSFC . The Cleanroom methodology 
emphasizes human discipline in program verification to pro- 
duce reliable software products that are "right the first 
time." 

Preliminary analysis of the Cleanroom case study shows that 
the method can be applied successfully in the GSFC Flight 
Dynamics Division (FDD), environment and may increase staff 
productivity and product quality. Compared to typical SEL 
activities, there is evidence of lower failure rates, a more 
complete and consistent set of inline code documentation, a 
different distribution of phase effort activity, and a dif- 
ferent growth profile in terms of lines of code developed. 
NTIS 
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SECTION 8 - ADA TECHNOLOGY 
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8.1 ADA TRAINING EVALUATION AND RECOMMENDATIONS FROM THE 

GAMMA RAY OBSERVATORY ADA DEVELOPMENT TEAM . SEL-85-002, 
R. MURPHY AND 'M. STARK, OCTOBER 1985, 41 PAGES 

This document presents the Ada training experiences of the 
Gamma Ray Observatory Ada development team and recommenda- 
tions for future Ada training of software developers. 
Training methods are evaluated; deficiencies in the training 
program are noted; and a recommended approach, including 
course outline, time allocation, and reference materials, is 
offered. NTIS 
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8.2 "DESIGNING WITH ADA FOR SATELLITE SIMULATION: A CASE 

STUDY, " W. W. AGRESTI, V. E. CHURCH, D. N. CARD, AND 
P. L. LO, PROCEEDINGS OF THE FIRST INTERNATIONAL 
SYMPOSIUM ON ADA FOR THE NASA SPACE STATION . JUNE 1986, 
14 PAGES 

This paper compares a FORTRAN-oriented and an Ada-oriented 
design for the same system to learn whether an essentially 
different design was produced using Ada. The designs were 
produced by an experiment that involves the parallel devel- 
opment of software for a spacecraft dynamics simulator. 
Design differences are identified in the use of abstrac- 
tions, system structure, and simulator operations. Although 
the designs were significantly different, this result may be 
influenced by some special characteristics discussed in the 
paper. JAO 


i 


This technical paper also appears in SEL-86-004, Collected 
Software Engineering Papers: Volume IV . November 1986. 
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8.3 "TOWARDS A GENERAL OBJECT-ORIENTED SOFTWARE DEVELOPMENT 
METHODOLOGY," E. SEIDEWITZ AND M. STARK, PROCEEDINGS 
OF THE FIRST INTERNATIONAL SYMPOSIUM ON ADA FOR THE 
NASA SPACE STATION . JUNE 1986, 14 PAGES 

This paper presents an overview of a general approach that 
allows a designer to apply powerful, object-oriented prin- 
ciples to a wide range of applications at all stages of 
design. Object-oriented design is the technique of using 
objects (abstract software models of problem domain enti- 
ties) as the basic units of modularity in system design. 

This method is discussed in terms of the identification of 
objects, object diagrams, the principles of abstractions and 
information hiding, and hierarchies (parent-child, and 
seniority). This paper also considers how object-oriented 
designs fits into the overall software life-cycle. JAO 



This technical paper also appears in SEL-86-004, Collected 
Software Engineering Papers: Volume IV . November 1986. 
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8.4 GENERAL OBJECT-ORIENTED SOFTWARE DEVELOPMENT . 

SEL-86-002, E. SEIDEWITZ AND M. STARK, AUGUST 1986, 

79 PAGES 

This report describes a general approach to object-oriented 
design, which synthesizes the principles of previous object- 
oriented methods into a unified framework. Further, this 
approach fits into the overall software life-cycle, provid- 
ing transitions from specification to design and from design 
to code. It, therefore, provides the basis for a general 
object-oriented development methodology. NTIS 


\ 
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8.5 "TOWARDS A GENERAL OBJECT-ORIENTED ADA LIFECYLE, " 

M. STARK AND E. SEIDEWITZ, PROCEEDINGS OF THE JOINT ADA 
CONFERENCE . MARCH 1987, 10 PAGES 

This paper provides a distillation of our experiences with 
object-oriented software development. It considers the use 
of entity-relationship and object data-flow techniques for 
an object-oriented specification, which leads smoothly into 
our design and implementation methods as well as an object- 
oriented approach to reusability in Ada. JAO 


This paper also appears in SEL-87-009,. Collected Software 
Engineering Papers: Volume V , November 1987. 


8978 


8-6 


8.6 "A STRUCTURE COVERAGE TOOL FOR ADA™ SOFTWARE SYSTEMS," 
L. WU, V. R. BASILI, AND K. REED, PROCEEDINGS OF THE 
JOINT ADA CONFERENCE . MARCH 1987, 9 PAGES 

This paper proposes a number of coverage measures for Ada 
features such as packages, generic units, and tasks, and 
discusses their interpretation in relation to the tradi- 
tional coverage metrics. It also proposes a mechanism for 
collecting these coverage measures. In addition, this paper 
suggests that coverage metrics may also be interpreted as 
indicators of dynamic system performance. JAQ 


This paper also appears in SEL-87-009, Collected Software 
Engineering Papers: Volume V . November 1987. 
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8.7 "TAME: TAILORING AN ADA™ MEASUREMENT ENVIRONMENT," 

V. R. BASILI AND H. D. ROMBACH, PROCEEDINGS OF THE JOINT 
ADA CONFERENCE . MARCH 1987, 7 PAGES 

This paper presents and discusses the TAME (Tailoring an Ada 
Measurement Environment) project, which aims at the develop- 
ment of a prototype measurement and evaluation environment 
that supports activities including the process of setting up 
measurement and evaluation goals and deriving supportive 
measures. We discuss the TAME requirements and architec- 
tural design, the status of the first prototype, and the 
expected impact of this project on Ada projects and APSEs. 

The prototype currently under development does not interface 
with an APSE; however, it is designed for being integrated 
into an APSE in the future. JAO 


This technical paper also appears in SEL-87-009, Collected 
Software Engineering Papers: Volume V . November 1987. 
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"LESSONS LEARNED IN USE OF ADA™-OR I ENTED DESIGN 
METHODS," C. E. BROPHY, W. W. AGRESTI , and V. R. BASILI 
PROCEEDINGS OF THE JOINT ADA CONFERENCE . MARCH 1987 
5 PAGES 

This paper describes a Flight Dynamics division (at NASA 
Goddard Space Flight Center) study that analyzes the effects 
of Ada on the development of their software. This project 
is one of the first to use Ada in this environment. In the 
study, two teams are each developing satellite simulators 
from the same specifications, one in Ada and one in FORTRAN, 
the standard language in this environment. This paper ad- 
dresses the lessons learned during the design phase, includ- 
ing the effect of specifications on Ada-oriented design, the 
importance of the design method chosen, the importance of 
the documentation style for the chosen design method, and 
the effects of Ada-oriented design on the software develop- 
ment life cycle. It is hoped that the issues faced in this 
project will show more clearly what may be expected in de- 
signing with Ada-oriented design methods. JAO 


This paper also appears in SEL-87-009, Collected Software 
Engineering Papers: Volume V . November 1987. 
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8.9 ADA® STYLE GUIDE (VERSION 1.1) . SEL-87-002, 

E. SEIDEWITZ , MAY 1987, 90 PAGES 

This document provides guidance on the appropriate use of 
Ada's powerful features. The Goddard Space Flight Center 
Ada User's Group has produced this style guide, which ad- 
dresses "program style" issues. The guide covers three 
areas of Ada program style: 

• The structural decomposition of a program. 

® The coding and use of specific Ada features, 
o The textual formatting of a program. NTIS 
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8.10 ASSESSING THE ADA® DESIGN PROCESS AND ITS IMPLICATIONS: 

A CASE STUDY . SEL-87-004, C. BROPHY AND S. GODFREY, JULY 
1987, 45 PAGES 

This document presents the results of a case study to ana- 
lyze the approach taken and the lessons learned during the 
design of the Gamma Ray Observatory Dynamics Simulator in 
Ada (GRODY) . Included are recommendations for defining the 
design phase and outlining the products that should be 
developed during this phase of the software development life 
cycle for future flight dynamics software systems developed 
in Ada. CAS I 
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8.11 "ASAP: AN ADA STATIC SOURCE CODE ANALYZER PROGRAM," 

D. L. DOUBLEDAY, UNIVERSITY OF MARYLAND, TECHNICAL. 

MEMORANDUM, AUGUST 1987, 100 PAGES 

This paper describes and provides a user's manual for ASAP, 
an automated tool for static source code analysis of pro- 
grams written in the Ada programming language. The purpose 
of the analysis is to collect and store information pertain- 
ing to the analyzed Ada compilation unit's size, complexity, 
usage of Ada language constructs and features, and static 
interface with other Ada compilation units. NTIS 
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"OBJECT-ORIENTED PROGRAMMING IN SMALLTALK AND ADA," 

E. SEIDEWITZ , PROCEEDINGS OF THE 1987 CONFERENCE ON 
OBJECT-ORIENTED PROGRAMMING SYSTEMS. LANGUAGES AND 
APPLICATIONS . OCTOBER 1987, 12 PAGES 

This paper compares the capabilities of modular languages 
such as Ada and Modula-2 with an archetypal object-oriented 
language such as Smalltalk. The comparison in this paper i 
in terms of the basic properties of encapsulation, inherit- 
ance, and binding, with examples given in both languages. 
This comparison highlights the strengths and weaknesses of 
both types of languages from an object-oriented perspective 
It also provides a basis for the application of experience 
from Smalltalk and other object-oriented languages to 
increasingly widely used modular languages such as Ada and 
Modula-2. JAO 


This technical paper also appears in SEL-88-002, Collected 
Software Engineering Papers: Volume VI . November 1988. 


8978 


8-13 


8.13 "MEASURING ADA FOR SOFTWARE DEVELOPMENT IN THE SOFT- 
WARE ENGINEERING LABORATORY (SEL) , " F. E. McGARRY AND 
W. W. AGRESTI, PROCEEDINGS OF THE 2 1ST ANNUAL HAWAII 
INTERNATIONAL CONFERENCE ON SYSTEM SCIENCES . JANUARY 
1988 , 9 PAGES 

This paper presents a SEL study of the parallel development 
of a production flight dynamics system by two teams of pro- 
fessional programmers. Both teams worked from the same set 
of requirements, with one team required to use the normal 
development process (FORTRAN), while the second team used 
the Ada development language. Detailed data were collected 
during the development phases to support the analysis. A 
discussion of the experimental approach and some of the key 
results from early, completed studies are presented. JAO 


This paper also appears in SEL-87-009, Collected Software 
Engineering Papers: Volume V . November 1987. 
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8.14 


"GENERAL OBJECT-ORIENTED SOFTWARE DEVELOPMENT: BACK- 

GROUND AND EXPERIENCE," E. SEIDEWITZ, PROCEEDINGS OF 
THE 2 1ST HAWAII INTERNATIONAL CONFERENCE ON SYSTEM 
SCIENCES . JANUARY 1988, 9 PAGES 

The effective use of Ada requires the adoption of modern 
software-engineering techniques such as object-oriented 
methodologies. A Goddard Space Flight Center Software Engi- 
neering Laboratory Ada pilot project has provided an oppor- 
tunity for studying object-oriented design in Ada. The 
project involves the development of a simulation system in 
Ada in parallel with a similar FORTRAN development. As part 
of the project, the Ada development team trained and eval- 
uated object-oriented and process-oriented design method- 
ologies for Ada. Finding these methodologies limited in 
various ways, the team created a general object-oriented 
development methodology which they applied to the project. 
This paper discusses some background on the development of 
the methodology, describes the main principles of the ap- 
proach and presents some experiences with using the method- 
ology, including a general comparison of the Ada and FORTRAN 
simulator designs. JAO 


This paper also appears in SEL-87-009, Collected Software 
Engineering Papers: Volume V . November 1987. 
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8.15 


"LESSONS LEARNED IN THE IMPLEMENTATION PHASE OF A 
LARGE ADA PROJECT," C. E. BROPHY, S. GODFREY, 

W. W. AGRESTI, AtfD V. R. BASILI, PROCEEDINGS OF THE 
WASHINGTON ADA TECHNICAL CONFERENCE . MARCH 1988, 

8 PAGES 

This paper discusses lessons learned during the implementa- 
tion phase of an ongoing Ada project in the Flight Dynamics 
Division of the NASA/Goddard Space Flight Center. It is 
part of a series of lessons-learned documents being written 
for each development phase. Topics which are discussed 
include (1) use of nesting versus library units, (2) code 
reading,. (3) unit testing, and (4) lessons learned using 
special Ada features. JAO 


This technical paper also appears in SEL-88-002, Collected 
Software Engineering Papers: Volume VI . November 1988. 
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8.16 "GENERAL OBJECT-ORIENTED SOFTWARE DEVELOPMENT WITH ADA: 
A LIFE CYCLE APPROACH," E. SEIDEWITZ, PROCEEDINGS OF 
THE CASE TECHNOLOGY CONFERENCE . APRIL 1988, 15 PAGES 

This paper discusses the advantages of using object-oriented 
concepts throughout the entire Ada software life cycle. The 
information presented here was obtained from an experience 
with the development of a simulation system in Ada. The 
paper considers the use of entity-relationship and process/ 
data-flow techniques for an object-oriented specification 
that leads smoothly into design and implementation methods, 
as well as an object-oriented approach to reusability in 
Ada . JAO 


This technical paper also appears in SEL-88-002, Collected 
Software Engineering Papers: Volume VI . November 1988. 
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8.17 "EXPERIENCES IN THE IMPLEMENTATION OF A LARGE ADA 
PROJECT / " S. GODFREY AND C. BROPHY , PROCEEDINGS OF 
THE 1988 WASHINGTON ADA SYMPOSIUM . JUNE 1988, 7 PAGES 

This paper discusses some of the similarities and differ- 
ences between the implementation of an Ada project and the 
implementation of a FORTRAN project. During the past several 
years, the Software Engineering Laboratory (SEL) of Goddard 
Space Flight Center has been conducting an experiment in Ada 
to determine the cost effectiveness and feasibility of using 
Ada to develop flight dynamics software and to assess the 
effect of Ada on the flight dynamics environment. This ex- 
periment consists of near-parallel developments of a dynamics 
simulator in both FORTRAN and Ada. A study team consisting 
of members from the SEL has monitored development progress 
and has collected data on both projects throughout their de- 
velopment. This paper compares the two projects and dis- 
cusses some of the interesting lessons learned during the 
code/unit test and integration phases of the Ada project. 

JAO 


This technical paper also appears in SEL-88-002, Collected 
Software Engineering Papers: Volume VI . November 1988. 
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8.18 SYSTEM TESTING OF A PRODUCTION ADA PROJECT - THE 
GRODY STUDY . SEL-88-001, J. SEIGLE AND Y. SHI, 
NOVEMBER 1988, 26 PAGES 

This paper discusses the impact that the Ada language and 
design methodologies that utilize its features have on the 
system test phase of the software development project life 
cycle. It is one of a series of lessons-learned reports 
examining each development phase. It evaluates the impact 
of the use of Ada when compared to FORTRAN in the system 
test phases of two projects. The paper concludes that 
although planning for system testing and conducting system 
tests were not generally affected by the use of Ada, the 
solving of problems found in system testing was generally 
facilitated by Ada constructs and design methodology. Most 
problems found in system testing were not due to difficulty 
with the language or methodology but rather due to lack of 
experience with the application. CAS I 
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8.19 EVOLUTION OF ADA TECHNOLOGY IN THE FLIGHT DYNAMICS 
AREA: DESIGN PHASE ANALYSIS . SEL-88-003, K. QUIMBY 

AND L. ESKER, DECEMBER 1988, 65 PAGES 

The software engineering issues related to the use of the 
Ada programming language during the design phase of an Ada 
project are analyzed. Discussion shows how an evolving 
understanding of these issues is reflected in the design 
processes of three "generations" of Ada projects. CAS I 
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8.20 PROCEEDINGS OF THE FIRST NASA ADA USER'S SYMPOSIUM . 
SEL-88-005 , DECEMBER 1988, 225 PAGES 

This document reproduces the presentations made by partici- 
pants at the First NASA Ada User's Symposium held in 
December 1988 at GSFC. The presentations were grouped into 
the following Ada categories: 

• Experiences 

• Applications 

• Directions and Implications 

The document also includes a summary of an open discussion 
among panelists. The following are the key points discussed 
in the summary: 

• Transition 

• Methodology 

• Training 

• Reuse 

• Real-Time 

CAS I 
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8.21 IMPLEMENTATION OF A PRODUCTION ADA PROJECT: THE 

GRODY STUDY . SEL-89-002, S. GODFREY AND C. BROPHY, 

MAY 1989 , 125 PAGES 

The SEL conducted an experiment in parallel development of 
two flight dynamics systems in FORTRAN and Ada. This docu- 
ment describes the differences observed during the imple- 
mentation, unit testing, and integration phases of the two 
projects and outlines the lessons learned during the imple- 
mentation phase of the Ada development. Included are rec- 
ommendations for future Ada development projects. CAS I 
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8.22 "EVOLUTION OF ADA TECHNOLOGY IN A PRODUCTION SOFTWARE 
ENVIRONMENT," F. McGARRY, L. ESKER, AND K. QUIMBY, 
PROCEEDINGS OF THE WASHINGTON ADA SYMPOSIUM (WADAS) . 
JUNE 1989, 9 PAGES 

The SEL performs studies and measurement related to evolving 
software technologies. The studies are aimed at understand- 
ing both the software development process and the impacts 
that evolving software practices may have on the software 
process and product. The SEL has conducted over 65 experi- 
ments by applying selected techniques to specific develop- 
ment efforts and measuring the resulting process and product. 

This paper analyzes the findings of an effort the SEL ini- 
tiated in early 1985 to study the characteristics, applica- 
tions, and impacts of Ada. Beginning with a relatively 
small practice problem (6,000 source lines of Ada), the SEL 
has collected detailed development data from a total of 
eight Ada projects (some of which are ongoing). The proj- 
ects range in size from 6,000 lines to approximately 
160,000 lines of code. JAO 


This technical paper also appears in SEL-89-006, Collected 
Software Engineering Papers: Volume VII . November 1989. 


8978 


8-23 


8.23 "USING ADA TO MAXIMIZE VERBATIM SOFTWARE REUSE," 

M. STARK AND E. BOOTH, PROCEEDINGS OF TRI-ADA 1989 . 

OCTOBER 1989, 13 PAGES 

This paper presents the lessons learned on several simulator 
projects in the Flight Dynamics Division (FDD) environment 
that exploit features of the Ada language, such as packages 
and generics, to achieve verbatim reuse. Verbatim reuse 
means that no changes are made to the component. These 
simulators are divided into two separate, but related, prob- 
lem domains, dynamics simulators, and telemetry simulators. 
FDD began using Ada in 1985 with the development of the 
Gamma Ray Observatory attitude dynamics simulator (GRODY) . 
Since that time, six additional simulator projects have been 
started. With each successive project, a concentrated ef- 
fort is made to use the lessons learned from previous Ada 
simulator development projects. 

This paper focuses on the concepts used in the projects that 
have had the most impact on verbatim software reuse in the 
FDD environment: GRODY, the Upper Atmosphere Research 

Satellite Telemetry Simulator, (UARSTELS) , and the Generic 
Dynamics and Telemetry Simulator (GENSIM) . This paper de- 
fines underlying design principles, discusses how Ada fea- 
tures support these principles for reuse in the small, and 
shows how these principles are used to achieve reuse in the 
large. Finally, this paper presents supporting data from 
current reusability results. JAO 


This technical paper also appears in SEL-89-006, Collected 
Software Engineering Papers: Volume VII , November 1989. 


8978 


8-24 


8.24 EVOLUTION OF ADA TECHNOLOGY IN THE FLIGHT DYNAMICS 
AREA; TMPLEMENTATION/TESTING PHASE ANALYSIS . 
SEL-89-004, K. QUIMBY, L. ESKER, L. SMITH, M. STARK, 
AND F. McGARRY, NOVEMBER 1989, 100 PAGES 

This document presents an analysis of the software engineer- 
ing issues related to the use of Ada for the implementation 
and systems testing phases of four Ada projects developed in 
the flight dynamics area. These projects reflect an evolv- 
ing understanding of more effective use of Ada features. In 
addition, the testing methodology used on these projects has 
changed substantially from that used on previous FORTRAN 
projects. CAS I 
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8.25 LESSONS LEARNED IN THE TRANSITION TO ADA FROM FORTRAN 
AT NASA/GODDARD . SEL-89-0Q5. C. BROPHY, NOVEMBER 1989, 
90 PAGES 

This document describes a case study performed at GSFC, in 
which two dynamics satellite simulators were developed from 
the same requirements, one in Ada and the other in FORTRAN. 
The purpose of the research was to find out how well the 
prescriptive Ada development model worked to develop the Ada 
simulator. The FORTRAN simulator development, as well as 
past FORTRAN developments, provided a baseline for compari- 
son. Since this was the first simulator developed here, the 
prescriptive Ada development model had many similarities to 
the usual FORTRAN development model. However, it was modi- 
fied to include longer design and shorter testing phases, 
which is generally expected with Ada developments. This 
document provides the results of the study as well as rec- 
ommendations for future Ada project development. NTIS 
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8.26 


"SOFTWARE RECLAMATION: IMPROVING POST- DEVELOPMENT 

REUSABILITY," J. BAILEY AND V. BASILI, PROCEEDINGS OF 
THE EIGHTH ANNUAL NATIONAL CONFERENCE ON ADA TECH- 
NOLOGY . MARCH 1990, 15 PAGES 

This paper describes part of a multi-year study of software 
reuse. It explores Ada program transformation techniques 
that preserve function but result in more independent and, 
therefore, presumably more reusable program components. 

Goals include a precise specification of the transform tech- 
nique and its application in a large development organiza- 
tion. Expected results are the identification of reuse 
promoters and inhibiters both in the problem space and in 
the solution space; the development of a set of metrics that 
can be applied to both the developing and completed software 
to reveal the expected degree of reusability; and the devel- 
opment of guidelines for both software developers and re- 
viewers that can help assure the desired reusability. 

The advantages of transforming existing software into reus- 
able components, rather than creating reusable components as 
an independent activity, include: (1) Using archives of 

previous projects that can yield reusable components, thus 
eliminating risk or development overhead, (2) Accomplishing 
transformation work in parallel with line developments but 
separately funded (particularly advantageous when developing 
software for an outside customer unwilling to sustain the 
additional costs and risks of reusable code development), 

(3) Creating components guaranteed to be relevant to the 
application area, and (4) Assuring low and controllable cost 
JAO 


This technical paper also appears in SEL-90-005, Collected 
Software Engineering Pacers: Volume VIII . November 1990. 
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8.27 


"ON DESIGNING PARAMETRIZED SYSTEMS USING ADA," 

M. STARK, PROCEEDINGS OF THE SEVENTH WASHINGTON 
ADA SYMPOSIUM . JUNE 1990, 7 PAGES 

This technical paper introduces the concept of a parametrized 
( reconf igurable) system. The Generic Simulator prototyping 
project (GENSIM) and planned Combined Operational Mission 
Planning and Attitude Support System (COMPASS) are discussed 
as illustrations of reconf igurable systems. A reuse model 
that supports parametrized systems is introduced, an example 
system is designed, and the direction of future work is dis- 
cussed. JAO 


This technical paper also appears in SEL-90-005, Collected 
Software Engineering Papers: Volume VIII . November 1990. 
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8.28 


"PUC: A FUNCTIONAL SPECIFICATION LANGUAGE FOR ADA, “ 

P. STRAUB AND M. ZELKOWITZ, PROCEEDINGS OF THE TENTH 
INTERNATIONAL CONFERENCE OF THE CHILEAN COMPUTER 
SCIENCE SOCIETY . JULY 1990, 13 PAGES 

This technical paper presents PUC, a specification language 
for Ada that addresses programmers * concerns for understand- 
ability. PUC is a functional language whose syntax and data 
types resemble Ada's, although it has features like polymor- 
phism and high order functions. This paper shows the re- 
quirements for the language PUC, presents an overview of the 
language and how it is used in the specification of Ada pro- 
grams, and gives the requirements and strategies for a semi- 
automatic translator from PUC to Ada. JAO 


This technical paper also appears in SEL-90-005, Collected 
Software Engineering Papers: Volume VIII . November 1990. 
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8.29 PROCEEDINGS OF THE SECOND NASA ADA USERS 1 SYMPOSIUM . 
SEL-89-008 , NOVEMBER 1989, 308 PAGES 

This document reproduces the presentations made by the par- 
ticipants at the Second NASA Ada Users' Symposium held in 
November 1989 at GSFC. The presentations were grouped into 
the following Ada categories: 

• NASA-Wide Activities 

• Center and Project Activities 

® Space Station Activities 

The document also includes a summary of the presentations 
and audience remarks. Approximately 380 people, represent- 
ing 75 private corporations, 3 universities, 17 agencies of 
the Federal Government, and 7 NASA centers, attended the 
symposium. NTIS 
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8.30 A STUDY OF THE PORTABILITY OF AN ADA SYSTEM IN THE 
SOFTWARE ENGINEERING LABORATORY (SEL) . SEL-9 0-003, 

L. JUN ET AL., JUNE 1990, 75 PAGES 

This document discusses a particular porting effort and 
gives statistics on the portability of Ada and the total 
effort required to accomplish the rehost, comparing this 
effort to past experiments on the rehosting of FORTRAN sys- 
tems. Also included is an analysis of the types of errors 
encountered during rehosting, the changes required to rehost 
the system, experiences with the Alsys IBM Ada compiler, the 
impediments encountered, and the lessons learned during the 
study. SDB 
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8.31 GAMMA RAY OBSERVATORY DYNAMICS SIMULATOR IN ADA 

( GRQDY ) EXPERIMENT SUMMARY . SEL-90-004, T. MCDERMOTT 
ET AL., SEPTEMBER 1990, 74 PAGES 

This document presents the final report on the Gamma Ray- 
Observatory (GRO) Dynamics Simulator in Ada (GRODY) experi- 
ment. The experiment consisted of the parallel development 
of dynamics simulators for GRO, one in FORTRAN and the other 
in Ada, to evaluate the applicability of Ada in the NASA/GSFC 
Flight Dynamics environment. Also evaluated by the Ada team 
were training approaches, an Ada methodology appropriate to 
the Flight Dynamics environment, and a baseline for evaluat- 
ing future Ada projects. SDB 
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8.32 "OBJECT-ORIENTED PROGRAMMING THROUGH TYPE EXTENSIONS 

IN ADA 9X,“ E. SEIDEWITZ , ADA LETTERS . 

MARCH/APRIL 1991, 12 PAGES 

This paper discusses several object-oriented programming 
features (inheritance, polymorphism, and dynamic binding) 
and proposes methods to incorporate them into the new Ada 
language standard. These additions would build on the 
existing Ada 83 features of typing and encapsulation without 
violating the current language design philosophy. To 
support inheritance and dynamic binding, syntax is presented 
to declare unconstrained types as extendible; extended types 
would inherit operations from the parent type and the 
operations on subtypes of extendible types would be 
dynamically overloaded. The paper proposes syntax to define 
subtypes so that one single subprogram defined on a parent 
type could act on all subtypes without requiring overloaded 
subprograms for each type, thus supporting the feature of 
polymorphism. To retain the benefits of polymorphism and 
dynamic binding along with encapsulation, the paper 
introduces a "private extension" mechanism, which allows 
private types to be extended subtypes. The author 
recognizes that other issues exist and must be resolved in a 
complete design for object-oriented features in Ada, but 
believes that inclusion of these features would be a start. 
JAO 


This technical paper also appears in SEL-91-005, Collected 
Software Engineering Papers: Volume IX . November 1991. 
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8.33 "AN OBJECT-ORIENTED APPROACH TO PARAMETERIZED SOFTWARE 
IN ADA," E. SEIDEWITZ AND M. STARK, PROCEEDINGS OF THE 
EIGHTH WASHINGTON ADA SYMPOSIUM . JUNE 1991, 15 PAGES 

This paper discusses techniques for specifying and 
implementing parameterized software systems and the 
distinctions between parameterized systems and other 
object-oriented development. It introduces the Combined 
Operational Mission Planning and Attitude Support System 
(COMPASS) , a parameterized system being developed by the 
GSFC Flight Dynamics Division (FDD). COMPASS is a large- 
scale system that covers three related domains. First, the 
use of domain analysis in the COMPASS specification approach 
is presented. The specification concepts are strongly 
object oriented. Objects, classes, categories, and 
subsystems are defined and discussed. The configuration of 
domain objects into a specific application is also 
addressed. A considerable amount of domain analysis has 
been done on COMPASS using these specification concepts. 
Second, the Ada implementation concepts planned for COMPASS 
(including class packages, module packages, and 
configuration) are presented. To date, these concepts have 
been applied only to a small amount of prototype code. 
However, both the specification and implementation concepts 
are firmly based on experience with Ada, generic simulation 
components, and object-oriented methodology, and have been 
tailored specifically for parameterized systems. The 
authors believe that the concepts applied on COMPASS, though 
still evolving, can provide a comprehensive approach to 
specifying and implementing parameterized systems in Ada. 

JAO 


This technical paper also appears in SEL-91-005, Collected 
Software Engineering Papers: Volume IX . November 1991. 
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8.34 SOFTWARE ENGINEERING LABORATORY ( SEL) ADA PERFORMANCE 
STUDY REPORT . SEL-91-003, E. W. BOOTH AND M. E. STARK, 
JULY 1991, 67 PAGES 

This document presents the goals, methods, and results of 
the Ada Performance Study. The goals and scope of the study 
as well as the background of Ada development in the Flight 
Dynamics Division (FDD) are discussed. The organization, 
purpose, methods, and results of each test are detailed, and 
analyses of the test results are provided. The document 
concludes with guidelines for future development efforts 
based on the analysis of results from this Ada Performance 
Study. An explanation of the approach used on the 
performance tests is included in the appendix. NTIS 
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8.35 "DESIGNING CONFIGURABLE SOFTWARE: COMPASS IMPLEMENTA- 

TION CONCEPTS," E. W. BOOTH AND M. E. STARK, PROCEED- 
INGS OF TRI-ADA 1991 . OCTOBER 1991, 12 PAGES 

This paper presents an overview of the Combined Operational 
Mission Planning and Attitude Support System (COMPASS) and 
discusses COMPASS implementation concepts, COMPASS prototyp- 
ing status, and the prototype team. The introduction briefly 
describes the development history of spacecraft attitude 
simulators in the GSFC Flight Dynamics Division (FDD) using 
object-oriented methods and Ada and the dramatic improvements 
in software reuse achieved by the use of generic architec- 
tures. The FDD is applying these successful reuse strategies 
on COMPASS, the largest Ada software system to be developed 
by the FDD. The COMPASS implementation concepts are based 
on the layered software reuse model and the use of Ada gen- 
erics. This paper describes the COMPASS detailed implemen- 
tation concepts associated with each of the following three 
major layers (from the lowest to the highest) and the two 
levels under each layer: 

• Services layer (system-independent and 
system-dependent levels) 

« Problem domain layer (domain language and domain 
classes levels) 

• Application layer (application modules and 
application architecture components levels) 

Each level uses and builds on the functionality provided by 
lower levels. Levels are not allowed to use or depend on 
functionality provided at higher levels. The generic unit 
feature of the Ada language allows parameterization of 
component dependencies within and between these levels. JAO 


This technical paper also appears in SEL-91-005, Collected 
Software Engineering Papers: Volume IX . November 1991. 
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SECTION 9 - DATA COLLECTION 
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9.1 "OPERATIONAL ASPECTS OF A SOFTWARE MEASUREMENT FACIL- 
ITY," M. V. ZELKOWITZ AND V. R. BASILI, PROCEEDINGS OF 
THE SOFTWARE LIFE CYCLE MANAGEMENT WORKSHOP . SEPTEMBER 
1977, 11 PAGES 

This technical paper describes the data collection opera- 
tions of the SEL. The source of the data is a group of 
flight dynamics software development projects at GSFC. The 
paper describes seven data collection forms used by the SEL. 
The procedure for transferring data from the forms to the 
computer database is outlined. Some of the validity checks 
performed on the data are identified. This procedure pro- 
duces valid, relevant data with which significant software 
engineering research can be conducted. JAO 
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9.2 NASA/SEL DATA COMPENDIUM . C. TURNER, G. CARON, AND 
G. BREMENT, DATA & ANALYSIS CENTER FOR SOFTWARE, 

SPECIAL PUBLICATION, MAY 1981, 80 PAGES 

This document summarizes the software engineering data col- 
lected by the SEL from flight dynamics development projects. 
A series of charts and graphs is presented for each project. 
These materials provide profiles of the development histo- 
ries of the projects and define some of the major character- 
istics of the software development process at GSFC. However 
this document is a factual presentation rather than an anal- 
ysis of the data. SDB 
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9.3 A COMPARISON OF RADC AND NASA/SEL SOFTWARE DEVELOPMENT 
DATA . C. TURNER AND G. CARON, DATA St ANALYSIS CENTER 
FOR SOFTWARE, MAY 1981, 31 PAGES 

This document reports the results of an analysis of the re- 
lationship between project size and several other software 
measures. These measures include productivity, effort, du- 
ration, errors, and error rate. The analysis used data from 
two sources: Rome Air Development Center (RADC) and the 

SEL. Least-squares regression techniques were applied to 
both sets of data. Results obtained from the two sets of 
data were comparable. The conclusion cited in the report is 
that RADC and SEL data can be combined, in most cases, to 
obtain a larger sample without undesirable side effects. SDB 
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9.4 AUTOMATED COLLECTION OF SOFTWARE ENGINEERING DATA IN 

THE SOFTWARE ENGINEERING LABORATORY ( SEL ) . SEL-81-014, 

A. L. GREEN, W. J. DECKER, AND F. E. McGARRY, 

SEPTEMBER 1981, 72 PAGES 

This document presents the results of an analysis of SEL data 
collection procedures. The principal questions addressed are 
what current manual procedures could be automated and how 
these automated procedures could be incorporated in the SEL 
database system. The functional requirements of such a sys- 
tem are identified and explained. The automatable sources of 
data identified in this report include the following: 

• Computer accounting information 

• Requirements language tools 

• Program design language tools 

o Programmer workbench features 

9 Source code analyzer program 

This document was also issued as Computer Sciences Corpora- 
tion document CSC/TM-8 1/6222 . CAST 
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9.5 GUIDE TO DATA COLLECTION . SEL-81-101, V. E. CHURCH, 

D. N. CARD, AND F. E. McGARRY, AUGUST 1982, 123 PAGES 

This document presents guidelines and recommendations for 
collecting software development data. The guide describes 
the motivation, planning, implementation, and management of 
a data collection effort. Other topics covered include 
types, sources, and availability of data; methods and costs 
of data collection; types of analyses supported; and warn- 
ings and suggestions based on SEL experience. The appen- 
dixes include facsimiles of SEL data collection forms and a 
glossary of software engineering terms. 

This document, abstracted and generalized from 5 years of 
SEL data collection experience, is intended to be a practi- 
cal guide, for software managers and engineers. It was also 
issued as Computer Sciences Corporation document 
CSC/TM-82/6 13 7 . NTIS 


The previous version of this document was Guide to Data 
Collection . SEL-81-001, V. E. Church, D. N. Card, and 
F. E. McGarry, September 1981. 
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"DATA COLLECTION AND EVALUATION FOR EXPERIMENTAL 
COMPUTER SCIENCE RESEARCH , " M. V. ZELKOWITZ , EMPIRICAL 
FOUNDATIONS FOR COMPUTER AND INFORMATION SCIENCE 
(PROCEEDINGS), NOVEMBER 1982, 15 PAGES 

This technical paper reviews the data collection procedures 
of the SEL and shows how they are used to generate data with 
one particular software engineering experiment. The SEL 
collects process and product measures from actual scientific 
software projects using a combination of automated tools and 
questionnaires. A project currently in progress, from which 
data are being collected, is a software prototyping effort. 
The goal of this experiment is to determine the costs and 
benefits of developing a prototype before developing a full 
system. JAO 


This technical paper also appears in SEL-83-003, Collected 
Software Engineering Papers: Volume II . November 1983. 
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9.7 A METHODOLOGY FOR COLLECTING VALID SOFTWARE ENGINEERING 
DATA . V. R. BASILI AND D. M. WEISS, UNIVERSITY OF 
MARYLAND, TECHNICAL REPORT, DECEMBER 1982, 22 PAGES 

This technical report describes an effective data collection 
method for evaluating software methodologies and for study- 
ing the software development process. The purpose of the 
report is to show how to obtain valid data that may be used 
both to learn more about the software development process 
and to evaluate software development methodologies in a pro- 
duction environment. The data collected during the study 
describes changes made to the software during development 
and is obtained when the changes are made. To ensure ac- 
curacy of the data, validation is performed concurrently 
with software development as part of the data collection 
process. Validation is based on interviews with the pro- 
grammers supplying the data. The feasibility of the data 
collection methodology was demonstrated by applying it to 
five difference project in two different production environ- 
ments. SDB 


This technical paper also appears in SEL-83-003, Collected 
Software Engineering Papers: Volume'-II , November 1983. 
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9.8 "A METHODOLOGY FOR COLLECTING VALID SOFTWARE ENGINEER- 
ING DATA, " V. R. BASILI AND D. M. WEISS, IEEE TRANS- 
ACTIONS ON SOFTWARE ENGINEERING . NOVEMBER 1984, 

11 PAGES 

This paper describes an effective data collection method for 
evaluating software development methodologies and studying 
the software development process. The method uses goal- 
directed data collection to evaluate methodologies with 
respect to the claims made for them. Such claims are used 
as a basis for defining the goals of the data collection, 
establishing a list of questions of interest to be answered 
by data analysis, defining a set of data categorization 
schemes, and designing a data collection form. 

The data to be collected are based on the changes made to 
the software during development and are obtained when the 
changes are made. To ensure accuracy of the data, valida- 
tion is performed concurrently with software development and 
data collection. Validation is based on interviews with 
those people supplying the data. Results from using the 
methodology show that data validation is a necessary part of 
change data collection. Without it, as much as 50 percent 
of the data may be erroneous. 

Feasibility of the data collection methodology was demon- 
strated by applying it to five different projects in two 
different environments. The application showed that the 
methodology was both feasible and useful. JAO 


This technical paper also appears in SEL-85-003, Collected 
Software Engineering Papers: Volume III . November 1985. 
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9.9 DATA COLLECTION PROCEDURES FOR THE REHOSTED SEL DATA- 
BASE . SEL-87-008, G. HELLER, OCTOBER 1987, 64 PAGES 


This document presents SEL-monitored flight dynamics soft- 
ware development project procedures for collecting data in 
support of software engineering research activities. An 
overview of data collection during the life cycle of a de- 
velopment project is presented. This overview is followed 
by a discussion of SEL measurement of the structure and 
growth of the software product. Finally, detailed instruc- 
tions for the completion and submission of SEL data collec- 
tion forms are presented. NTIS 
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9.10 SOFTWARE ENGINEERING LABORATORY ( SEL) DATABASE 

ORGANIZATION AND USER'S GUIDE (REVISION 1) . SEL-89-101, 
M. SO ET AL., FEBRUARY 1990, 230 PAGES 

This document presents the organization of the SEL data- 
base. Included are definitions and detailed descriptions of 
the database tables and views, the SEL data, and system 
support data. The mapping from the SEL and system support 
data to the base tables is described. In addition, tech- 
niques for accessing the database through the Database Ac- 
cess Manager for the SEL (DAMSEL) system and via the ORACLE 
structured query language (SQL) are discussed. CAS I 


The previous version of this document was SEL Data Base 
Organization and User's Guide . SEL-89-001, M. So. et al., 
April 1989. 
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9.11 DATABASE ACCESS MANAGER FOR THE SOFTWARE 

ENGINEERING LABORATORY ( DAMSEL ) USER'S GUIDE . 
SEL-90-001, M. BUHLER ET AL. , MARCH 1990, 

338 PAGES 

This document presents step-by-step operational instructions 
for the Database Access Manager for the Software Engineering 
Laboratory (DAMSEL) system. It includes detailed instruc- 
tions for performing various data entry and report generation 
activities with sample sessions showing the user interface 
display screens and output for each of the reports. In 
addition, it groups the available functions by the classes 
of users that may access them. CAST 
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APPENDIX - OTHER REFERENCES 


The following is a list of the references that are no longer 
included in the Annotated Bibliography of SEL Literature. 

• The Software Engineering Laboratory . SEL-77-001, 

V. R. Basili, M. V. Zelkowitz, F. E. McGarry, 

et al./ May 1977 — Superseded by The Software Engi- 
neering Laboratory . SEL-81-104, D. N. Card, 

F. E. McGarry, G. Page, et al., February 1982 

• "Analysis Software Requirements for the Data 
Retrieval System," D. N. Card and V. E. Church, 
Computer Sciences Corporation, Technical 
Memorandum, March 1983 — Software superseded 

• "Configuration Analysis Tool Design," F. K. Banks, 
Computer Sciences Corporation, Technical Memoran- 
dum, March 1980 — Software superseded 
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